idnits 2.17.1 draft-bishop-quic-http-and-qpack-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC7540], [RFC7541], [I-D.ietf-quic-transport]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (December 14, 2017) is 2325 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-34) exists of draft-ietf-quic-http-07 == Outdated reference: A later version (-34) exists of draft-ietf-quic-transport-07 ** Obsolete normative reference: RFC 7540 (Obsoleted by RFC 9113) Summary: 2 errors (**), 0 flaws (~~), 4 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 QUIC Working Group M. Bishop 3 Internet-Draft Akamai 4 Intended status: Standards Track December 14, 2017 5 Expires: June 17, 2018 7 Header Compression for HTTP/QUIC 8 draft-bishop-quic-http-and-qpack-07 10 Abstract 12 HTTP/2 [RFC7540] uses HPACK [RFC7541] for header compression. 13 However, HPACK relies on the in-order message-based semantics of the 14 HTTP/2 framing layer in order to function. Messages can only be 15 successfully decoded if processed by the decoder in the same order as 16 generated by the encoder. This draft refines HPACK to loosen the 17 ordering requirements for use over QUIC [I-D.ietf-quic-transport]. 19 Status of This Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF). Note that other groups may also distribute 26 working documents as Internet-Drafts. The list of current Internet- 27 Drafts is at https://datatracker.ietf.org/drafts/current/. 29 Internet-Drafts are draft documents valid for a maximum of six months 30 and may be updated, replaced, or obsoleted by other documents at any 31 time. It is inappropriate to use Internet-Drafts as reference 32 material or to cite them other than as "work in progress." 34 This Internet-Draft will expire on June 17, 2018. 36 Copyright Notice 38 Copyright (c) 2017 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (https://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with respect 46 to this document. Code Components extracted from this document must 47 include Simplified BSD License text as described in Section 4.e of 48 the Trust Legal Provisions and are provided without warranty as 49 described in the Simplified BSD License. 51 Table of Contents 53 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 54 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 55 2. QPACK - Concepts . . . . . . . . . . . . . . . . . . . . . . 3 56 2.1. Changes to Static and Dynamic Tables . . . . . . . . . . 4 57 2.1.1. Dynamic Table State Synchronization . . . . . . . . . 4 58 2.2. Encoding Constraints . . . . . . . . . . . . . . . . . . 6 59 2.2.1. Permitted References . . . . . . . . . . . . . . . . 6 60 2.2.2. Header Table Size . . . . . . . . . . . . . . . . . . 6 61 3. Wire Format . . . . . . . . . . . . . . . . . . . . . . . . . 7 62 3.1. Feedback Stream . . . . . . . . . . . . . . . . . . . . . 8 63 3.1.1. HEADERS_DONE . . . . . . . . . . . . . . . . . . . . 8 64 3.1.2. ACK_FLUSH . . . . . . . . . . . . . . . . . . . . . . 8 65 3.1.3. DROP . . . . . . . . . . . . . . . . . . . . . . . . 9 66 3.1.4. ACK_DROP . . . . . . . . . . . . . . . . . . . . . . 9 67 3.2. Checkpoint Streams . . . . . . . . . . . . . . . . . . . 10 68 3.2.1. INSERT . . . . . . . . . . . . . . . . . . . . . . . 10 69 3.2.2. TOUCH . . . . . . . . . . . . . . . . . . . . . . . . 12 70 3.3. Request Streams . . . . . . . . . . . . . . . . . . . . . 12 71 3.3.1. Indexed Header Field Representation . . . . . . . . . 13 72 3.3.2. Literal Header Field Representation . . . . . . . . . 13 73 4. Use in HTTP/QUIC . . . . . . . . . . . . . . . . . . . . . . 14 74 4.1. SETTING_QPACK_BLOCKING_PERMITTED . . . . . . . . . . . . 15 75 4.2. SETTING_QPACK_INITIAL_CHECKPOINT . . . . . . . . . . . . 15 76 5. Implementation trade-offs . . . . . . . . . . . . . . . . . . 15 77 5.1. Compression Efficiency versus Blocking Avoidance . . . . 16 78 5.2. Timely State Transitions versus Decoder Complexity . . . 16 79 6. Security Considerations . . . . . . . . . . . . . . . . . . . 17 80 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 81 7.1. Settings . . . . . . . . . . . . . . . . . . . . . . . . 17 82 7.2. Errors . . . . . . . . . . . . . . . . . . . . . . . . . 18 83 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 18 84 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 18 85 9.1. Normative References . . . . . . . . . . . . . . . . . . 18 86 9.2. Informative References . . . . . . . . . . . . . . . . . 19 87 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 19 89 1. Introduction 91 HPACK has a number of features that were intended to provide 92 performance advantages to HTTP/2, but which don't live well in an 93 out-of-order environment such as that provided by QUIC. 95 The largest challenge is the fact that elements are referenced by a 96 very fluid index. Not only is the index implicit when an item is 97 added to the header table, the index will change without notice as 98 other items are added to the header table. Static entries occupy the 99 first 61 values, followed by dynamic entries. A newly-added dynamic 100 entry would cause older dynamic entries to be evicted, and the 101 retained items are then renumbered beginning with 62. This means 102 that, without processing all preceding header sets, no index into the 103 dynamic table can be interpreted, and the index of a given entry 104 cannot be predicted. 106 Any solution to the above will almost certainly fall afoul of the 107 memory constraints the decompressor imposes. The automatic eviction 108 of entries is done based on the compressor's declared dynamic table 109 size, which MUST be less than the maximum permitted by the 110 decompressor (and relayed using an HTTP/2 SETTINGS value). 112 Further, streams in QUIC are lossy in the presence of stream resets. 113 While HTTP/2 (via TCP) guarantees the delivery of all previously-sent 114 data on a stream even if that stream is reset, QUIC does not 115 retransmit lost frames if a stream has been reset, and may discard 116 data which has not yet been delivered to the application. 118 Early versions of QPACK were small deltas of HPACK to introduce 119 order-resiliency. Recent versions depart from HPACK more 120 substantially to add resilience against reset message streams and 121 reduce the impact of head-of-line blocking. 123 In the following sections, this document proposes a successor to 124 HPACK which makes different trade-offs, enabling partial out-of-order 125 interpretation and bounded memory consumption with minimal head-of- 126 line blocking. None of the proposed improvements to HPACK (strongly- 127 typed fields, binary compression of common header syntax) are 128 currently included, but certainly could be. 130 1.1. Terminology 132 In this document, the key words "MUST", "MUST NOT", "REQUIRED", 133 "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", 134 and "OPTIONAL" are to be interpreted as described in BCP 14, 135 [RFC2119] and indicate requirement levels for compliant 136 implementations. 138 2. QPACK - Concepts 140 HPACK combines header table modification and message header emission 141 in a single sequence of coded bytes. QPACK bifurcates these into 142 three channels: 144 o Connection-wide sets of table update instructions sent on non- 145 request streams 147 o Connection-wide feedback on stream and checkpoint state on a 148 single non-request stream 150 o Non-modifying instructions which use the current header table 151 state to encode message headers on request streams 153 Because the per-message instructions introduce no changes to the 154 header table state, no state is lost if these instructions are 155 discarded due to a stream reset. Because the updates to the header 156 table supply their own order controls (the checkpoint logic), they 157 can be processed in any order and therefore delivered as messages 158 using unidirectional QUIC streams. 160 2.1. Changes to Static and Dynamic Tables 162 QPACK uses two tables for associating header fields to indexes. The 163 static table is unchanged from [RFC7541]. Unlike in [RFC7541], the 164 tables are not concatenated, but are referenced separately. 166 The dynamic table is a map from index to header field. Indices are 167 arbitrary numbers between 1 and 2^27. Each insert instruction will 168 specify the index being modified. While any index MAY be chosen for 169 a new entry, smaller numbers will yield better compression 170 performance. 172 With decoder consent (see Section 4.1), it is possible for QPACK 173 instructions to arrive which reference indices which have not yet 174 been defined. Such instructions MUST wait until the index definition 175 has arrived. In order to guard against malicious peers, 176 implementations supporting blocking SHOULD impose a time limit and 177 treat expiration of the timer as a decoding error. 179 2.1.1. Dynamic Table State Synchronization 181 In order to ensure table consistency, all modifications of the header 182 table occur as separate messages rather than on request streams. 183 Request streams contain only indexed and literal header entries. 185 No entries are automatically evicted from the dynamic table. Size 186 management is purely the responsibility of the encoder, which MUST 187 NOT exceed the declared memory size of the decoder. 189 To simplify state management in the dynamic table, _checkpoints_ are 190 introduced. A checkpoint is used to track entries added to the 191 dynamic table and streams that reference those entries, rather than 192 maintaining the full state of which streams reference which table 193 entries. 195 Checkpoints are unordered and have an identifier which MUST be unique 196 among checkpoints which have not been dropped. Each checkpoint has a 197 unidirectional stream which begins with its identifier and contains a 198 series of updates associated with that checkpoint. These updates 199 SHOULD be processed as they arrive; it is not necessary (and might 200 not be desirable) to wait for all instructions associated with a 201 checkpoint to arrive before beginning to process it. 203 The feedback stream is used to relay state transitions to the peer. 204 For example, when a decoder is done processing a header block, it 205 signals this using the HEADERS_DONE message. The encoder uses this 206 information to track which checkpoints can be dropped. 208 2.1.1.1. Checkpoint Lifecycle 210 A checkpoint is created by opening a new checkpoint stream. This 211 places the checkpoint in the NEW state for both encoder and decoder. 212 The encoder typically has at least one checkpoint in the NEW state. 214 Flushing a checkpoint is a two-step operation. First, the checkpoint 215 stream is closed. At that time, the encoder's NEW checkpoint becomes 216 PENDING. The decoder moves its NEW checkpoint directly to LIVE and 217 responds with an ACK_FLUSH message on the feedback stream. When the 218 encoder receives this message, its PENDING checkpoint becomes LIVE. 220 Unused entries are evicted indirectly, by dropping checkpoints. 221 Before a checkpoint can be dropped, its state is changed to DYING. 222 Changing a checkpoint's state to DYING allows the checkpoint to age 223 out. This is a strictly internal state on the encoder, and not 224 visible to the decoder. A DYING checkpoint can be returned to LIVE 225 at the encoder's discretion if necessary. 227 The encoder can change a DYING checkpoint to DEAD (sending a DROP 228 instruction) when it is no longer referenced by any outstanding 229 header blocks. The encoder sends the DROP command to the decoder 230 when it declares a checkpoint DEAD. 232 To ensure consistency, the decoder drops the corresponding checkpoint 233 and responds with an ACK_DROP message only when it has fully received 234 all instructions the encoder has issued up to that point. The 235 encoder drops the DEAD checkpoint upon receipt of the ACK_DROP 236 message. 238 When a checkpoint is dropped by encoder or decoder, the table entries 239 it references are checked: if an entry is no longer referenced by any 240 checkpoint, the entry is evicted. 242 Dropping a checkpoint and the entries associated with it is not 243 limited to just the oldest checkpoint; any DYING checkpoint - as long 244 as state transition rules are followed - may be dropped. This 245 flexibility permits the encoder to use a number of strategies for 246 entry eviction. 248 As long as the maximum dynamic table size is observed, new 249 checkpoints can be created; no upper limit on the number of 250 checkpoints is specified. A well-balanced spread of checkpoints 251 permits the encoder to recycle entries effectively. 253 2.2. Encoding Constraints 255 2.2.1. Permitted References 257 When encoding headers on a request stream, an encoder MAY reference 258 any static table entry or any dynamic header table entry referenced 259 by a LIVE checkpoint. References to entries in NEW or PENDING 260 checkpoints are permitted only if the client has set 261 "SETTING_QPACK_BLOCKING_PERMITTED" (see Section 4.1). 263 If a decoder receives a reference to an empty slot in the dynamic 264 table but has not sent "SETTING_QPACK_BLOCKING_PERMITTED", this MUST 265 be treated as a stream error of type "ERROR_QPACK_INVALID_REFERENCE" 266 if on a request stream. References to empty slots in the dynamic 267 table on a checkpoint stream MUST be treated as a connection error of 268 type "ERROR_QPACK_INVALID_REFERENCE". 270 References to DYING checkpoints are possible by returning the 271 checkpoint to LIVE, but this is usually inadvisable. Table entries 272 contained only in a DEAD checkpoint can never be referenced. 274 2.2.2. Header Table Size 276 As in HPACK, the dynamic table is constrained to the maximum size 277 specified by the decoder. An attempt to add a header to the dynamic 278 table or to create a new checkpoint which causes it to exceed the 279 maximum size MUST be treated as an error by a decoder. To enable 280 encoders to reclaim space, encoders can drop old checkpoints (see 281 Section 2.1.1). 283 The total table size is calculated as follows: 285 o The size of each entry is calculated as in HPACK 286 o Each checkpoint that has not been removed, regardless of state, 287 consumes 64 bytes 289 2.2.2.1. Table Size Changes 291 HTTP/QUIC prohibits mid-stream changes of settings. As a result, 292 only one table size change is possible: From the value a client 293 assumes during the 0-RTT flight to the actual value included in the 294 server's SETTINGS frame. The assumed value is required to be either 295 a server's previous value or zero. A server whose configuration has 296 recently changed MAY overlook inadvertent violations of its maximum 297 table size during the first round-trip. 299 In the case that the value has increased, either from zero to a non- 300 zero value or from the cached value to a higher value, no action is 301 required by the client. The encoder can simply begin using the 302 additional space. In the case that the value has decreased, the 303 encoder MUST move checkpoints to the DYING state which, upon removal, 304 would bring the table within the required size. 306 Regardless of changes to header table size, the encoder MUST NOT 307 create new checkpoints or add entries to the table which would result 308 in a size greater than the maximum permitted. This can imply that no 309 additions are permitted while waiting for old checkpoints to 310 complete. 312 3. Wire Format 314 QPACK instructions occur on three stream types, each of which uses a 315 separate instruction space. 317 The feedback stream is a bidirectional server-initiated stream used 318 for acknowledgement of actions and checkpoint state management. 319 Checkpoint streams are unidirectional streams from encoder to 320 decoder. Both types of streams consist of a series of QPACK 321 instructions with no message boundaries, preceded by a stream header 322 for checkpoint streams. 324 Finally, the contents of HEADERS and PUSH_PROMISE frames on request 325 streams reference the QPACK table state. 327 This section describes the instructions which are possible on each 328 stream type. 330 3.1. Feedback Stream 332 Stream 1, the first server-initiated bidirectional stream, is used as 333 the feedback stream, since the client does not need to begin sending 334 data on this stream until it has received data from the server. 336 This stream is critical to the HTTP/QUIC connection, and carries a 337 stream of the instructions defined in this section. Data on this 338 stream SHOULD be processed as soon as it arrives. 340 3.1.1. HEADERS_DONE 342 When the decoder has processed a frame containing header emission 343 instructions (Section 3.3, HEADERS or PUSH_PROMISE frames) on a 344 stream, it MUST emit a HEADERS_DONE message on the feedback stream. 345 The same Stream ID can be identified multiple times, as multiple 346 header-containing blocks can be sent on a single stream in the case 347 of intermediate responses, trailers, pushed requests, etc. 349 Since header frames on a request stream are received and processed in 350 order, this gives the encoder precise feedback on which header blocks 351 within a stream have been fully processed. This information can then 352 be used to correctly track outstanding stream references to 353 checkpoints. 355 0 1 2 3 4 5 6 7 356 +---+---+---+---+---+---+---+---+ 357 | 1 | Stream ID (7+) | 358 +---+---------------------------+ 360 HEADERS_DONE instruction 362 3.1.2. ACK_FLUSH 364 When the decoder has finished processing all instructions that make 365 up a checkpoint, it MUST indicate successful processing to the 366 encoder by emitting an ACK_FLUSH instruction on the feedback stream. 368 Upon emitting an ACK_FLUSH, the checkpoint transitions from NEW to 369 LIVE on the decoder. Upon receipt of an ACK_FLUSH, the checkpoint 370 transitions from PENDING to LIVE on the encoder. 372 0 1 2 3 4 5 6 7 373 +---+---+---+---+---+---+---+---+ 374 | 0 | 1 | 0 | Checkpoint ID (5+)| 375 +---+---------------------------+ 377 ACK_FLUSH instruction 379 3.1.3. DROP 381 When an encoder has received sufficient HEADERS_DONE messages to know 382 that a DYING checkpoint has no outstanding references, it emits a 383 DROP instruction to inform the decoder that the checkpoint can be 384 removed. Upon sending a DROP instruction, a DYING checkpoint becomes 385 DEAD. The DROP instruction also includes the IDs of any PENDING or 386 NEW checkpoints which reference entries contained in the checkpoint 387 being dropped. The "L" bit in each byte indicates whether another 388 checkpoint ID follows (L=0) or this is the final byte of the DROP 389 instruction (L=1). 391 Upon receiving a DROP instruction, if all listed checkpoints have 392 been fully processed (transitioned from NEW to LIVE), the identified 393 LIVE checkpoint is immediately removed from the decoder state and an 394 ACK_DROP instruction is emitted. Otherwise, the decoder saves the 395 DROP instruction until other checkpoints become LIVE. 397 0 1 2 3 4 5 6 7 398 +---+---+---+---+---+---+---+---+ 399 | 0 | 0 | L | Checkpoint ID (5+)| 400 +---+---+---+-------------------+ 401 | L | Checkpoint (7+) | 402 +---+---------------------------+ 403 | L | Checkpoint (7+) | 404 +---+---------------------------+ 405 | ... | 406 +-------------------------------+ 408 DROP instruction 410 3.1.4. ACK_DROP 412 When a decoder receives a DROP instruction, it removes the referenced 413 checkpoint from its state and clears any table entries which were 414 referenced only by that checkpoint. It then emits an ACK_DROP 415 instruction. When an encoder receives an ACK_DROP instruction, it 416 removes the corresponding DEAD checkpoint from its state and clears 417 any table entries which were referenced only by that checkpoint. 419 0 1 2 3 4 5 6 7 420 +---+---+---+---+---+---+---+---+ 421 | 0 | 1 | 1 | Checkpoint ID (5+)| 422 +---+---+---+-------------------+ 424 ACK_DROP instruction 426 3.2. Checkpoint Streams 428 Each checkpoint stream indicates the creation and content of a NEW 429 checkpoint. Each checkpoint has an ID; these IDs are chosen 430 arbitrarily by the encoder, though lower values SHOULD be preferred. 431 IDs of checkpoints which have been dropped MAY be reused for future 432 NEW checkpoints. 434 When the encoder has finished writing all data on the stream, it 435 changes the checkpoint to PENDING. When the decoder has received and 436 processed all data on the stream, it changes the checkpoint to LIVE 437 and generates an ACK_FLUSH. 439 Unidirectional streams in HTTP/QUIC begin with a stream header 440 indicating the nature of the stream content; the identifier for QPACK 441 checkpoints is 0x4B. 443 *Note to readers:* This header does not currently exist in the 444 main draft, but has manifested in several PRs, and would need to 445 be resurrected. 447 Following the stream header, a checkpoint stream contains its 448 checkpoint ID as an 8-bit prefix integer. The remainder of the 449 stream's data consists of the instructions defined in this section. 451 Data on checkpoint streams SHOULD be processed as soon as it arrives. 452 If multiple checkpoint streams are received at once, a decoder SHOULD 453 process data on each as it arrives if it has sent 454 "SETTINGS_QPACK_BLOCKING_PERMITTED", but MAY process checkpoint 455 streams one at a time. 457 3.2.1. INSERT 459 An addition to the dynamic table starts with the '1' one-bit pattern, 460 followed by the new index of the header represented as an integer 461 with a 7-bit prefix. The decoder adds the supplied header to the 462 checkpoint currently being processed, which is in the NEW state. 464 If the header field name matches the header field name of an entry 465 stored in the static table or the dynamic table, the header field 466 name can be represented using the index of that entry. In this case, 467 the "S" bit indicates whether the reference is to the static (S=1) or 468 dynamic (S=0) table and the index of the entry is represented as an 469 integer with an 7-bit prefix (see Section 5.1 of [RFC7541]). This 470 value is always non-zero. 472 If an INSERT instruction uses an existing dynamic table entry for the 473 name of an entry being added to the NEW checkpoint, both the existing 474 entry and the new entry are referenced by the NEW checkpoint. INSERT 475 instructions which reference the dynamic table MUST reference only 476 entries which are already included in a LIVE checkpoint. This avoids 477 the possibility of one checkpoint stream blocking on a different 478 checkpoint. 480 0 1 2 3 4 5 6 7 481 +---+---+---+---+---+---+---+---+ 482 | 1 | New Index (7+) | 483 +---+---------------------------+ 484 | S | Name Index (7+) | 485 +---+---------------------------+ 486 | H | Value Length (7+) | 487 +---+---------------------------+ 488 | Value String (Length octets) | 489 +-------------------------------+ 491 INSERT instruction -- Indexed Name 493 Otherwise, the header field name is represented as a string literal 494 (see Section 5.2 of [RFC7541]). A value 0 is used in place of the 495 table reference, followed by the header field name. 497 0 1 2 3 4 5 6 7 498 +---+---+---+---+---+---+---+---+ 499 | 1 | New Index (7+) | 500 +---+---------------------------+ 501 | 0 | 502 +---+---------------------------+ 503 | H | Name Length (7+) | 504 +---+---------------------------+ 505 | Name String (Length octets) | 506 +---+---------------------------+ 507 | H | Value Length (7+) | 508 +---+---------------------------+ 509 | Value String (Length octets) | 510 +-------------------------------+ 512 INSERT instruction -- New Name 514 Either form of header field name representation is followed by the 515 header field value represented as a string literal (see Section 5.2 516 of [RFC7541]). 518 An encoder MUST NOT attempt to place a value at an index not known to 519 be vacant. A decoder MUST treat the attempt to insert into an 520 occupied slot or reference a name in a vacant slot as a fatal error. 522 3.2.2. TOUCH 524 This instruction is emitted to link a NEW checkpoint to an existing 525 header table entry created by a previous checkpoint. This causes the 526 entry not to be removed from the table so long as the current 527 checkpoint is alive. 529 0 1 2 3 4 5 6 7 530 +---+---+---+---+---+---+---+---+ 531 | 0 | Index (7+) | 532 +---+---------------------------+ 534 Indexed Header Field 536 The encoder SHOULD NOT issue multiple TOUCH commands for the same 537 entry in the context of the same NEW checkpoint. If a non-existent 538 index is specified, the decoder MUST treat is as an error. 540 3.3. Request Streams 542 Frames which carry HTTP message headers begin with an optional 543 preface indicating potentially-blocking references in the frame. If 544 present, this preface indicates that the request depends on one or 545 more checkpoints which were NEW or PENDING for the encoder when the 546 frame was generated. If these checkpoints are not LIVE on the 547 decoder, it MAY delay reading the remainder of the frame until they 548 are. (If any of these checkpoints have already been dropped, this 549 must be treated as a stream error of type 550 ERROR_QPACK_INVALID_REFERENCE.) 552 The preface is formatted as follows: 554 0 1 2 3 4 5 6 7 555 +---+---+---+---+---+---+---+---+ 556 | L | Checkpoint (7+) | 557 +---+---+---+-------------------+ 558 | L | Checkpoint (7+) | 559 +---+---------------------------+ 560 | ... | 561 +-------------------------------+ 563 QPACK preface 565 The "L" bit indicates that this checkpoint is the last checkpoint in 566 the preface; if the bit is unset (0), then another checkpoint 567 follows. 569 3.3.1. Indexed Header Field Representation 571 An indexed header field representation identifies an entry in either 572 the static table or the dynamic table and causes that header field to 573 be added to the decoded header list, as described in Section 3.2 of 574 [RFC7541]. 576 0 1 2 3 4 5 6 7 577 +---+---+---+---+---+---+---+---+ 578 | 1 | S | Index (6+) | 579 +---+---+-----------------------+ 581 Indexed Header Field 583 An indexed header field starts with the '1' 1-bit pattern, followed 584 by the "S" bit indicating whether the reference is into the static 585 (S=1) or dynamic (S=0) table. Finally, the index of the matching 586 header field is represented as an integer with a 6-bit prefix (see 587 Section 5.1 of [RFC7541]). 589 The index value of 0 is not used. It MUST be treated as a decoding 590 error if found in an indexed header field representation. 592 3.3.2. Literal Header Field Representation 594 A literal header field representation starts with the '0' 1-bit 595 pattern and causes a header field to be added the decoded header 596 list. 598 The second bit, 'N', indicates whether an intermediary is permitted 599 to add this header to the dynamic header table on subsequent hops. 600 When the 'N' bit is set, the encoded header MUST always be encoded 601 with this specific literal representation. In particular, when a 602 peer sends a header field that it received represented as a literal 603 header field with the 'N' bit set, it MUST use the same 604 representation to forward this header field. This bit is intended 605 for protecting header field values that are not to be put at risk by 606 compressing them (see Section 7.1 of [RFC7541] for more details). 608 If the header field name matches the header field name of an entry 609 stored in the static table or the dynamic table, the header field 610 name can be represented using the index of that entry. In this case, 611 the "S" bit indicates whether the reference is to the static (S=1) or 612 dynamic (S=0) table and the index of the entry is represented as an 613 integer with an 5-bit prefix (see Section 5.1 of [RFC7541]). This 614 value is always non-zero. 616 0 1 2 3 4 5 6 7 617 +---+---+---+---+---+---+---+---+ 618 | 0 | N | S | Name Index (5+) | 619 +---+---+---+-------------------+ 620 | H | Value Length (7+) | 621 +---+---------------------------+ 622 | Value String (Length octets) | 623 +-------------------------------+ 625 Literal Header Field -- Indexed Name 627 Otherwise, the header field name is represented as a string literal 628 (see Section 5.2 of [RFC7541]). A value 0 is used in place of the 629 6-bit index, followed by the header field name. 631 0 1 2 3 4 5 6 7 632 +---+---+---+---+---+---+---+---+ 633 | 0 | N | 0 | 634 +---+---+-----------------------+ 635 | H | Name Length (7+) | 636 +---+---------------------------+ 637 | Name String (Length octets) | 638 +---+---------------------------+ 639 | H | Value Length (7+) | 640 +---+---------------------------+ 641 | Value String (Length octets) | 642 +-------------------------------+ 644 Literal Header Field -- Literal Name 646 Either form of header field name representation is followed by the 647 header field value represented as a string literal (see Section 5.2 648 of [RFC7541]). 650 4. Use in HTTP/QUIC 652 HTTP/QUIC [I-D.ietf-quic-http] currently retains the HPACK encoder/ 653 decoder from HTTP/2, but restricts the size of the dynamic table to 654 zero. Using QPACK instead would entail the following changes: 656 o Header Blocks consist of QPACK data instead of HPACK data 658 o HEADERS and PUSH_PROMISE frames define a flag indicating the 659 presence of a preface. 661 o Just as unidirectional push streams have a stream header 662 identifying their Push ID, a header will need to be added to 663 differentiate checkpoint streams from pushes 665 o Stream 2 is reserved for the Feedback Stream 667 A HEADERS or PUSH_PROMISE frame MAY contain an arbitrary number of 668 QPACK instructions. A partial HEADERS or PUSH_PROMISE frame MAY be 669 processed upon arrival and the resulting partial header set emitted 670 or buffered according to implementation requirements. 672 4.1. SETTING_QPACK_BLOCKING_PERMITTED 674 An HTTP/QUIC implementation can trade off the complexity of its QPACK 675 decoder against compression efficiency by permitting the peer's 676 compressor to reference unacknowledged entries. In the case of loss 677 on a checkpoint stream, such references might cause the processing of 678 request streams to block, waiting for the arrival of missing data. 680 If the decoder permits the encoder to make blocking references, it 681 sets "SETTING_QPACK_BLOCKING_PERMITTED" (0xSETTING-TBD1) to a non- 682 zero value. The encoder receiving this setting MAY encode up to this 683 number of potentially-blocking references at a time. 685 Sending this setting with no value indicates that a decoder is 686 willing to tolerate blocking references bounded only by the allowed 687 number of streams. If a decoder does not send this setting or sends 688 this setting with a value of zero, the encoder MUST NOT encode a 689 header using a reference that might block. 691 4.2. SETTING_QPACK_INITIAL_CHECKPOINT 693 An HTTP/QUIC implementation MAY include the 694 "SETTING_QPACK_INITIAL_CHECKPOINT" (0xSETTING_TBD2) setting, 695 containing the full serialization of an initial checkpoint stream's 696 data. If present, this setting MUST be fully processed by the peer 697 before decoding any checkpoint streams or header frames on request 698 streams. 700 The checkpoint defined by this setting is considered LIVE by both the 701 encoder and the decoder from the beginning of the connection. The 702 decoder does not need to send an ACK_FLUSH message confirming receipt 703 of this setting. 705 5. Implementation trade-offs 707 This document specifies a means for the encoder to express the 708 choices it made while encoding, but intentionally does not mandate 709 what those choices should be. In this section, potential areas for 710 implementation tuning are explored. 712 5.1. Compression Efficiency versus Blocking Avoidance 714 If blocking references are permitted, they will block if the frame 715 containing the entry definition is lost or delayed. Encoders MAY 716 choose to trade off compression efficiency and avoid blocking by 717 using literal instructions rather than referencing the dynamic table 718 until the insertion is believed to be complete. 720 The most efficient compression algorithm will reference a table entry 721 whenever it exists in the table, but risks blocking when subject to 722 packet loss or reordering. The most conservative algorithm will 723 always emit literals to guarantee that no blocking will ever occur. 724 Most implementations will choose a balance between these two 725 extremes. 727 Better efficiency while being similarly conservative can be achieved 728 by permitting references to table entries only once these entries are 729 confirmed to be present in the table. More optimization can be 730 achieved when the reference is known to be in the same packet as the 731 definition. 733 Increases in efficiency can be achieved by assuming greater risk of 734 blocking - implementations might choose a particular balance, or 735 adjust their aggressiveness based on observed network 736 characteristics. 738 Since it is possible to insert header values without emitting them on 739 a stream, an encoder MAY also proactively insert header values which 740 it believes will be needed on future requests, at the cost of reduced 741 compression efficiency for incorrect predictions. 743 The ability to split updates to the header table into discrete 744 checkpoints reduces the possibility for head-of-line blocking within 745 the checkpoint streams. Implementations SHOULD limit the size of 746 checkpoints to avoid head-of-line blocking within these messages. 748 5.2. Timely State Transitions versus Decoder Complexity 750 Anything which prevent checkpoints from transitioning from DYING to 751 DEAD can prevent the encoder from adding any new entries due to the 752 maximum table size. This does not block the encoder from continuing 753 to make requests, but could sharply limit compression performance. 754 Encoders would be well-served to begin moving checkpoint to DYING in 755 advance of encountering the table maximum. Decoders SHOULD be prompt 756 about emitting STREAM_DONE and ACK_DROP instructions to enable the 757 encoder to recover the table space. 759 Similarly, for decoders which prohibit blocking references, delaying 760 the transition of a checkpoint from PENDING to LIVE will degrade 761 compression performance. Decoders SHOULD consume checkpoint data and 762 emit ACK_FLUSH frames as promptly as possible. 764 Since decoders cannot safely drop old checkpoints until they have 765 fully processed any checkpoints which might have been open 766 concurrently, a long-lived checkpoint can delay the completion of an 767 ACK_DROP. Encoders SHOULD flush all NEW checkpoints as soon as 768 feasible after issuing a DROP instruction. 770 6. Security Considerations 772 A malicious encoder might attempt to consume a large amount of space 773 on the decoder, but as each decoder chooses how much memory to allow 774 the peer to consume, this state is bounded. 776 A malicious encoder might also send blocking references to entries 777 which will never actually be defined. This attack is comparable to a 778 "slow loris" attack in which a request is delivered very slowly in an 779 attempt to consume resources on the server. Similar mitigations 780 (request timers, etc.) SHOULD be employed to guard against such 781 attacks. 783 7. IANA Considerations 785 This document registers two settings and one error code with the 786 corresponding HTTP/QUIC registries. 788 7.1. Settings 790 This document registers two entries in the "HTTP/QUIC Settings" 791 registry established by [I-D.ietf-quic-http]. 793 Setting Name: SETTING_QPACK_BLOCKING_PERMITTED 795 Code: 0xSETTING-TBD1 797 Specification: Section 4.1 799 and 801 Setting Name: SETTING_QPACK_INITIAL_CHECKPOINT 803 Code: 0xSETTING-TBD2 805 Specification: Section 4.2 807 7.2. Errors 809 This document registers one error code in the "HTTP/QUIC Error Code" 810 registry established by [I-D.ietf-quic-http]. 812 Error name: ERROR_QPACK_INVALID_REFERENCE 814 Code: 0xERROR-TBD 816 Description: A blocking reference was received by a decoder which 817 did not permit it 819 Specification: Section 2.2.1 821 8. Acknowledgements 823 This draft draws heavily on the text of [RFC7541], and adopts (with 824 adaptation) the checkpoint model from [QMIN]. The direct and 825 indirect input of those authors is gratefully acknowledged, as well 826 as ideas gleefully stolen from: 828 o Jana Iyengar 830 o Patrick McManus 832 o Martin Thomson 834 o Charles 'Buck' Krasic 836 o Kyle Rose 838 o Alan Frindell 840 A substantial portion of Mike's work on this draft was supported by 841 Microsoft during his employment there. 843 9. References 845 9.1. Normative References 847 [I-D.ietf-quic-http] 848 Bishop, M., "Hypertext Transfer Protocol (HTTP) over 849 QUIC", draft-ietf-quic-http-07 (work in progress), October 850 2017. 852 [I-D.ietf-quic-transport] 853 Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed 854 and Secure Transport", draft-ietf-quic-transport-07 (work 855 in progress), October 2017. 857 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 858 Requirement Levels", BCP 14, RFC 2119, 859 DOI 10.17487/RFC2119, March 1997, 860 . 862 [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext 863 Transfer Protocol Version 2 (HTTP/2)", RFC 7540, 864 DOI 10.17487/RFC7540, May 2015, 865 . 867 [RFC7541] Peon, R. and H. Ruellan, "HPACK: Header Compression for 868 HTTP/2", RFC 7541, DOI 10.17487/RFC7541, May 2015, 869 . 871 9.2. Informative References 873 [QMIN] Tikhonov, D., "QMIN: Header Compression for QUIC", draft- 874 tikhonov-quic-qmin-00 (work in progress), November 2017. 876 Author's Address 878 Mike Bishop 879 Akamai 881 Email: mbishop@evequefou.be