idnits 2.17.1 draft-ietf-httpbis-variants-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([2], [3], [4], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. -- The draft header indicates that this document updates RFC7234, but the abstract doesn't seem to mention this, which it should. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC7234, updated by this document, for RFC5378 checks: 2007-12-21) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (March 25, 2019) is 1831 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 842 -- Looks like a reference, but probably isn't: '2' on line 844 -- Looks like a reference, but probably isn't: '3' on line 846 -- Looks like a reference, but probably isn't: '4' on line 848 == Outdated reference: A later version (-19) exists of draft-ietf-httpbis-header-structure-09 ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7231 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7234 (Obsoleted by RFC 9111) == Outdated reference: A later version (-15) exists of draft-ietf-httpbis-client-hints-06 Summary: 4 errors (**), 0 flaws (~~), 3 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 HTTP M. Nottingham 3 Internet-Draft Fastly 4 Updates: 7234 (if approved) March 25, 2019 5 Intended status: Standards Track 6 Expires: September 26, 2019 8 HTTP Representation Variants 9 draft-ietf-httpbis-variants-05 11 Abstract 13 This specification introduces an alternative way to communicate a 14 secondary cache key for a HTTP resource, using the HTTP "Variants" 15 and "Variant-Key" response header fields. Its aim is to make HTTP 16 proactive content negotiation more cache-friendly. 18 Note to Readers 20 _RFC EDITOR: please remove this section before publication_ 22 Discussion of this draft takes place on the HTTP working group 23 mailing list (ietf-http-wg@w3.org), which is archived at 24 https://lists.w3.org/Archives/Public/ietf-http-wg/ [1]. 26 Working Group information can be found at https://httpwg.github.io/ 27 [2]; source code and issues list for this draft can be found at 28 https://github.com/httpwg/http-extensions/labels/variants [3]. 30 There is a prototype implementation of the algorithms herein at 31 https://github.com/mnot/variants-toy [4]. 33 Status of This Memo 35 This Internet-Draft is submitted in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF). Note that other groups may also distribute 40 working documents as Internet-Drafts. The list of current Internet- 41 Drafts is at https://datatracker.ietf.org/drafts/current/. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on September 26, 2019. 50 Copyright Notice 52 Copyright (c) 2019 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (https://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 68 1.1. Notational Conventions . . . . . . . . . . . . . . . . . 4 69 2. The "Variants" HTTP Header Field . . . . . . . . . . . . . . 5 70 2.1. Relationship to Vary . . . . . . . . . . . . . . . . . . 7 71 3. The "Variant-Key" HTTP Header Field . . . . . . . . . . . . . 7 72 4. Cache Behaviour . . . . . . . . . . . . . . . . . . . . . . . 9 73 4.1. Compute Possible Keys . . . . . . . . . . . . . . . . . . 10 74 4.2. Check Vary . . . . . . . . . . . . . . . . . . . . . . . 11 75 4.3. Example of Cache Behaviour . . . . . . . . . . . . . . . 11 76 4.3.1. A Variant Missing From the Cache . . . . . . . . . . 12 77 4.3.2. Variants That Don't Overlap the Client's Request . . 13 78 5. Origin Server Behaviour . . . . . . . . . . . . . . . . . . . 13 79 5.1. Examples . . . . . . . . . . . . . . . . . . . . . . . . 14 80 5.1.1. Single Variant . . . . . . . . . . . . . . . . . . . 14 81 5.1.2. Multiple Variants . . . . . . . . . . . . . . . . . . 15 82 5.1.3. Partial Coverage . . . . . . . . . . . . . . . . . . 15 83 6. Defining Content Negotiation Using Variants . . . . . . . . . 16 84 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 85 8. Security Considerations . . . . . . . . . . . . . . . . . . . 17 86 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 17 87 9.1. Normative References . . . . . . . . . . . . . . . . . . 17 88 9.2. Informative References . . . . . . . . . . . . . . . . . 18 89 9.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 18 90 Appendix A. Variants for Existing Content Negotiation Mechanisms 19 91 A.1. Accept . . . . . . . . . . . . . . . . . . . . . . . . . 19 92 A.2. Accept-Encoding . . . . . . . . . . . . . . . . . . . . . 20 93 A.3. Accept-Language . . . . . . . . . . . . . . . . . . . . . 20 94 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 21 95 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 21 97 1. Introduction 99 HTTP proactive content negotiation ([RFC7231], Section 3.4.1) is 100 seeing renewed interest, both for existing request headers like 101 Accept-Language and for newer ones (for example, see 102 [I-D.ietf-httpbis-client-hints]). 104 Successfully reusing negotiated responses that have been stored in a 105 HTTP cache requires establishment of a secondary cache key 106 ([RFC7234], Section 4.1). Currently, the Vary header ([RFC7231], 107 Section 7.1.4) does this by nominating a set of request headers. 109 HTTP's caching model allows a certain amount of latitude in 110 normalising those request header field values, so as to increase the 111 chances of a cache hit while still respecting the semantics of that 112 header. However, normalisation is not formally defined, leading to 113 divergence in cache behaviours. 115 Even when the headers' semantics are understood, a cache does not 116 know enough about the possible alternative representations available 117 on the origin server to make an appropriate decision. 119 For example, if a cache has stored the following request/response 120 pair: 122 GET /foo HTTP/1.1 123 Host: www.example.com 124 Accept-Language: en;q=0.5, fr;q=1.0 126 HTTP/1.1 200 OK 127 Content-Type: text/html 128 Content-Language: en 129 Vary: Accept-Language 130 Transfer-Encoding: chunked 132 [English content] 134 Provided that the cache has full knowledge of the semantics of 135 Accept-Language and Content-Language, it will know that an English 136 representation is available and might be able to infer that a French 137 representation is not available. But, it does not know (for example) 138 whether a Japanese representation is available without making another 139 request, incurring possibly unnecessary latency. 141 This specification introduces the HTTP Variants response header field 142 (Section 2) to enumerate the available variant representations on the 143 origin server, to provide clients and caches with enough information 144 to properly satisfy requests - either by selecting a response from 145 cache or by forwarding the request towards the origin - by following 146 the algorithm defined in Section 4. 148 Its companion Variant-Key response header field (Section 3) indicates 149 the applicable key(s) that the response is associated with, so that 150 it can be reliably reused in the future. When this specification is 151 in use, the example above might become: 153 GET /foo HTTP/1.1 154 Host: www.example.com 155 Accept-Language: en;q=0.5, fr;q=1.0 157 HTTP/1.1 200 OK 158 Content-Type: text/html 159 Content-Language: en 160 Vary: Accept-Language 161 Variants: Accept-Language;de;en;jp 162 Variant-Key: en 163 Transfer-Encoding: chunked 165 [English content] 167 Proactive content negotiation mechanisms that wish to be used with 168 Variants need to define how to do so explicitly; see Section 6. As a 169 result, it is best suited for negotiation over request headers that 170 are well-understood. 172 Variants also works best when content negotiation takes place over a 173 constrained set of representations; since each variant needs to be 174 listed in the header field, it is ill-suited for open-ended sets of 175 representations. 177 Variants can be seen as a simpler version of the Alternates header 178 field introduced by [RFC2295]; unlike that mechanism, Variants does 179 not require specification of each combination of attributes, and does 180 not assume that each combination has a unique URL. 182 1.1. Notational Conventions 184 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 185 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 186 "OPTIONAL" in this document are to be interpreted as described in BCP 187 14 [RFC2119] [RFC8174] when, and only when, they appear in all 188 capitals, as shown here. 190 This specification uses the Augmented Backus-Naur Form (ABNF) 191 notation of [RFC5234] but relies on Structured Headers from 192 [I-D.ietf-httpbis-header-structure] for parsing. 194 Additionally, it uses the "field-name" rule from [RFC7230], and 195 "type", "subtype", "content-coding" and "language-range" from 196 [RFC7231]. 198 2. The "Variants" HTTP Header Field 200 The Variants HTTP response header field indicates what 201 representations are available for a given resource at the time that 202 the response is produced, by enumerating the request header fields 203 that it varies on, along with the values that are available for each. 205 Variants is a Structured Header [I-D.ietf-httpbis-header-structure]. 206 Its value MUST be a list-of-lists (Section 3.3 of 207 [I-D.ietf-httpbis-header-structure]) whose members are strings 208 (Section 3.8 of [I-D.ietf-httpbis-header-structure]) or tokens 209 (Section 3.9 of [I-D.ietf-httpbis-header-structure]). Its ABNF is: 211 Variants = sh-list-of-lists 213 If Structured Header parsing fails or a list-member has the wrong 214 type, the client MUST treat the representation as having no Variants 215 header field. 217 The Variants header field represents an ordered list of "variant- 218 axes", each of which consists of a request header "field-name" string 219 and a list of "available-value" strings. Each inner-list in the 220 Variants header field value is parsed into a variant-axis. The first 221 list-member of the inner-list is interpreted as the field-name, and 222 the remaining list-members are the available-values. Any list-member 223 that is a token (Section 3.9 of [I-D.ietf-httpbis-header-structure]) 224 is interpreted as a string containing the same characters. 226 Field-names in the Variants header field value MUST match the field- 227 name production (Section 3.2 of [RFC7230]). Clients receiving an 228 invalid field-name MUST NOT match it to any content negotiating 229 mechanism. 231 So, given this example header field: 233 Variants: Accept-Encoding;gzip 235 a recipient can infer that the only content-coding available for that 236 resource is "gzip" (along with the "identity" non-encoding; see 237 Appendix A.2). 239 Given: 241 Variants: accept-encoding 243 a recipient can infer that no content-codings (beyond identity) are 244 supported. Note that as always, field-name is case-insensitive. 246 A more complex example: 248 Variants: Accept-Encoding;gzip;br, Accept-Language;en ;fr 250 Here, recipients can infer that two content-codings in addition to 251 "identity" are available, as well as two content languages. Note 252 that, as with all Structured Header lists, they might occur in the 253 same header field or separately, like this: 255 Variants: Accept-Encoding;gzip;brotli 256 Variants: Accept-Language;en ;fr 258 The ordering of available-values after the field-name is significant, 259 as it might be used by the header's algorithm for selecting a 260 response (in this example, the first language is the default; see 261 Appendix A.3). 263 The ordering of the request header fields themselves indicates 264 descending application of preferences; in the example above, a cache 265 that has all of the possible permutations stored will honour the 266 client's preferences for Accept-Encoding before honouring Accept- 267 Language. 269 Origin servers SHOULD consistently send Variant header fields on all 270 cacheable (as per [RFC7234], Section 3) responses for a resource, 271 since its absence will trigger caches to fall back to Vary 272 processing. 274 Likewise, servers MUST send the Variant-Key response header field 275 when sending Variants, since its absence means that the stored 276 response will not be reused when this specification is implemented. 278 _RFC EDITOR: Please remove the next paragraph before publication._ 280 Implementations of drafts of this specification MUST implement an 281 HTTP header field named "Variants-##" instead of the "Variants" 282 header field specified by the final RFC, with "##" replaced by the 283 draft number being implemented. For example, implementations of 284 draft-ietf-httpbis-variants-05 would implement "Variants-05". 286 2.1. Relationship to Vary 288 This specification updates [RFC7234] to allow caches that implement 289 it to ignore request header fields in the Vary header for the 290 purposes of secondary cache key calculation ([RFC7234], Section 4.1) 291 when their semantics are implemented as per this specification and 292 their corresponding response header field is listed in Variants. 294 If any member of the Vary header does not have a corresponding 295 variant that is understood by the implementation, it is still subject 296 to the requirements there. 298 See Section 5.1.3 for an example. 300 In practice, implementation of Vary varies considerably. As a 301 result, cache efficiency might drop considerably when Variants does 302 not contain all of the headers referenced by Vary, because some 303 implementations might choose to disable Variants processing when this 304 is the case. 306 3. The "Variant-Key" HTTP Header Field 308 The Variant-Key HTTP response header field identifies a set of 309 variants provided by the representation it occurs within. A variant 310 is identified by a selection of one available-value from each 311 variant-axis from the Variants header field. 313 Variant-Key is a Structured Header 314 [I-D.ietf-httpbis-header-structure]. Its value MUST be a list-of- 315 lists (Section 3.3 of [I-D.ietf-httpbis-header-structure]) whose 316 members are strings (Section 3.8 of 317 [I-D.ietf-httpbis-header-structure]) or tokens (Section 3.9 of 318 [I-D.ietf-httpbis-header-structure]). Its ABNF is: 320 Variant-Key = sh-list-of-lists 322 If Structured Header parsing fails or a list-member has the wrong 323 type, the client MUST treat the representation as having no Variant- 324 Key header field. 326 Each inner-list MUST have the same number of list-members as there 327 are variant-axes in the representation's Variants header field. If 328 not, the client MUST treat the representation as having no Variant- 329 Key header field. 331 Each list-member is treated as identifying an available-value for the 332 corresponding variant-axis' field-name. Any list-member that is a 333 token (Section 3.9 of [I-D.ietf-httpbis-header-structure]) is 334 interpreted as a string containing the same characters. These 335 available-values do not need to explicitly appear in the Variants 336 header field. For example, Accept-Encoding defines an implicit 337 "identity" available-value (Appendix A.2). 339 For example: 341 Variants: Accept-Encoding;gzip;br, Accept-Language;en ;fr 342 Variant-Key: gzip;fr 344 This header pair indicates that the representation has a "gzip" 345 content-coding and "fr" content-language. 347 If the response can be used to satisfy more than one request, they 348 can be listed in additional members. For example: 350 Variants: Accept-Encoding;gzip;br, Accept-Language;en ;fr 351 Variant-Key: gzip;fr, "identity";fr 353 indicates that this response can be used for requests whose Accept- 354 Encoding algorithm selects "gzip" or "identity", as long as the 355 Accept-Language algorithm selects "fr" - perhaps because there is no 356 gzip-compressed French representation. 358 When more than one Variant-Key value is in a response, the first one 359 present MUST correspond to the request that caused that response to 360 be generated. 362 Parsing is strict. For example: 364 Variants: Accept-Encoding;gzip;br, Accept-Language;en ;fr 365 Variant-Key: gzip;fr, identity;fr, br;fr;oops 367 is treated as if the Variant-Key header were completely absent, which 368 will tend to disable caching for the representation that contains it. 370 Note that in 372 Variant-Key: gzip ;fr 373 Variant-Key: "gzip ";fr 375 The whitespace after "gzip" in the first header field value is 376 excluded by the token parsing algorithm, but the whitespace in the 377 second header field value is included by the string parsing 378 algorithm. This will likely cause the second header field value to 379 fail to match client requests. 381 _RFC EDITOR: Please remove the next paragraph before publication._ 382 Implementations of drafts of this specification MUST implement an 383 HTTP header field named "Variant-Key-##" instead of the "Variant-Key" 384 header field specified by the final RFC, with "##" replaced by the 385 draft number being implemented. For example, implementations of 386 draft-ietf-httpbis-variants-05 would implement "Variant-Key-05". 388 4. Cache Behaviour 390 Caches that implement the Variants header field and the relevant 391 semantics of the field-names it contains can use that knowledge to 392 either select an appropriate stored representation, or forward the 393 request if no appropriate representation is stored. 395 They do so by running this algorithm (or its functional equivalent) 396 upon receiving a request: 398 Given incoming-request (a mapping of field-names to lists of field 399 values), and stored-responses (a list of stored responses suitable 400 for reuse as defined in Section 4 of [RFC7234], excepting the 401 requirement to calculate a secondary cache key): 403 1. If stored-responses is empty, return an empty list. 405 2. Order stored-responses by the "Date" header field, most recent to 406 least recent. 408 3. Let sorted-variants be an empty list. 410 4. If the freshest member of stored-responses (as per [RFC7234], 411 Section 4.2) has one or more "Variants" header field(s) that 412 successfully parse according to Section 2: 414 1. Select one member of stored-responses with a "Variants" 415 header field-value(s) that successfully parses according to 416 Section 2 and let variants-header be this parsed value. This 417 SHOULD be the most recent response, but MAY be from an older 418 one as long as it is still fresh. 420 2. For each variant-axis in variants-header: 422 1. If variant-axis' field-name corresponds to the request 423 header field identified by a content negotiation 424 mechanism that the implementation supports: 426 1. Let request-value be the field-value associated with 427 field-name in incoming-request (after being combined 428 as allowed by Section 3.2.2 of [RFC7230]), or null if 429 field-name is not in incoming-request. 431 2. Let sorted-values be the result of running the 432 algorithm defined by the content negotiation 433 mechanism with request-value and variant-axis' 434 available-values. 436 3. Append sorted-values to sorted-variants. 438 At this point, sorted-variants will be a list of lists, each 439 member of the top-level list corresponding to a variant-axis 440 in the Variants header field-value, containing zero or more 441 items indicating available-values that are acceptable to the 442 client, in order of preference, greatest to least. 444 5. Return result of running Compute Possible Keys (Section 4.1) on 445 sorted-variants, an empty list and an empty list. 447 This returns a list of lists of strings suitable for comparing to the 448 parsed Variant-Keys (Section 3) that represent possible responses on 449 the server that can be used to satisfy the request, in preference 450 order, provided that their secondary cache key (after removing the 451 headers covered by Variants) matches. Section 4.2 illustrates one 452 way to do this. 454 4.1. Compute Possible Keys 456 This algorithm computes the cross-product of the elements of key- 457 facets. 459 Given key-facets (a list of lists of strings), and key-stub (a list 460 of strings representing a partial key), and possible-keys (a list of 461 lists of strings): 463 1. Let values be the first member of key-facets. 465 2. Let remaining-facets be a copy of all of the members of key- 466 facets except the first. 468 3. For each value in values: 470 1. Let this-key be a copy of key-stub. 472 2. Append value to this-key. 474 3. If remaining-facets is empty, append this-key to possible- 475 keys. 477 4. Otherwise, run Compute Possible Keys on remaining-facets, 478 this-key and possible-keys. 480 4. Return possible-keys. 482 4.2. Check Vary 484 This algorithm is an example of how an implementation can meet the 485 requirement to apply the members of the Vary header field that are 486 not covered by Variants. 488 Given stored-response (a stored response): 490 1. Let filtered-vary be the field-value(s) of stored-response's 491 "Vary" header field. 493 2. Let processed-variants be a list containing the request header 494 fields that identify the content negotiation mechanisms supported 495 by the implementation. 497 3. Remove any member of filtered-vary that is a case-insensitive 498 match for a member of processed-variants. 500 4. If the secondary cache key (as calculated in [RFC7234], 501 Section 4.1) for stored_response matches incoming-request, using 502 filtered-vary for the value of the "Vary" response header, return 503 True. 505 5. Return False. 507 This returns a Boolean that indicates whether stored-response can be 508 used to satisfy the request. 510 Note that implementation of the Vary header field varies in practice, 511 and the algorithm above illustrates only one way to apply it. It is 512 equally viable to forward the request if there is a request header 513 listed in Vary but not Variants. 515 4.3. Example of Cache Behaviour 517 For example, if the selected variants-header was: 519 Variants: Accept-Language;en;fr;de, Accept-Encoding;gzip;br 521 and the request contained the headers: 523 Accept-Language: fr;q=1.0, en;q=0.1 524 Accept-Encoding: gzip 526 Then the sorted-variants would be: 528 [ 529 ["fr", "en"] // prefers French, will accept English 530 ["gzip", "identity"] // prefers gzip encoding, will accept identity 531 ] 533 Which means that the result of the Cache Behaviour algorithm would 534 be: 536 [ 537 ["fr", "gzip"], 538 ["fr", "identity"], 539 ["en", "gzip"], 540 ["en", "identity"] 541 ] 543 Representing a first preference of a French, gzip'd response. Thus, 544 if a cache has a response with: 546 Variant-Key: fr; gzip 548 it could be used to satisfy the first preference. If not, responses 549 corresponding to the other keys could be returned, or the request 550 could be forwarded towards the origin. 552 4.3.1. A Variant Missing From the Cache 554 If the selected variants-header was: 556 Variants: Accept-Language;en;fr;de 558 And a request comes in with the following headers: 560 Accept-Language: de;q=1.0, es;q=0.8 562 Then sorted-variants in Cache Behaviour is: 564 [ 565 ["de"] // prefers German; will not accept English 566 ] 568 If the cache contains responses with the following Variant-Keys: 570 Variant-Key: fr 571 Variant-Key: en 573 Then the cache needs to forward the request to the origin server, 574 since Variants indicates that "de" is available, and that is 575 acceptable to the client. 577 4.3.2. Variants That Don't Overlap the Client's Request 579 If the selected variants-header was: 581 Variants: Accept-Language;en;fr;de 583 And a request comes in with the following headers: 585 Accept-Language: es;q=1.0, ja;q=0.8 587 Then sorted-variants in Cache Behaviour are: 589 [ 590 ["en"] 591 ] 593 This allows the cache to return a "Variant-Key: en" response even 594 though it's not in the set the client prefers. 596 5. Origin Server Behaviour 598 Origin servers that wish to take advantage of Variants will need to 599 generate both the Variants (Section 2) and Variant-Key (Section 3) 600 header fields in all cacheable responses for a given resource. If 601 either is omitted and the response is stored, it will have the effect 602 of disabling caching for that resource until it is no longer stored 603 (e.g., it expires, or is evicted). 605 Likewise, origin servers will need to assure that the members of both 606 header field values are in the same order and have the same length, 607 since discrepancies will cause caches to avoid using the responses 608 they occur in. 610 The value of the Variants header should be relatively stable for a 611 given resource over time; when it changes, it can have the effect of 612 invalidating previously stored responses. 614 As per Section 2.1, the Vary header is required to be set 615 appropriately when Variants is in use, so that caches that do not 616 implement this specification still operate correctly. 618 Origin servers are advised to carefully consider which content 619 negotiation mechanisms to enumerate in Variants; if a mechanism is 620 not supported by a receiving cache, it will "downgrade" to Vary 621 handling, which can negatively impact cache efficiency. 623 5.1. Examples 625 The operation of Variants is illustrated by the examples below. 627 5.1.1. Single Variant 629 Given a request/response pair: 631 GET /clancy HTTP/1.1 632 Host: www.example.com 633 Accept-Language: en;q=1.0, fr;q=0.5 635 HTTP/1.1 200 OK 636 Content-Type: image/gif 637 Content-Language: en 638 Cache-Control: max-age=3600 639 Variants: Accept-Language;en;de 640 Variant-Key: en 641 Vary: Accept-Language 642 Transfer-Encoding: chunked 644 Upon receipt of this response, the cache knows that two 645 representations of this resource are available, one with a language 646 of "en", and another whose language is "de". 648 Subsequent requests (while this response is fresh) will cause the 649 cache to either reuse this response or forward the request, depending 650 on what the selection algorithm determines. 652 So, if a request with "en" in Accept-Language is received and its 653 q-value indicates that it is acceptable, the stored response is used. 654 A request that indicates that "de" is acceptable will be forwarded to 655 the origin, thereby populating the cache. A cache receiving a 656 request that indicates both languages are acceptable will use the 657 q-value to make a determination of what response to return. 659 A cache receiving a request that does not list either language as 660 acceptable (or does not contain an Accept-Language at all) will 661 return the "en" representation (possibly fetching it from the 662 origin), since it is listed first in the Variants list. 664 Note that Accept-Language is listed in Vary, to assure backwards- 665 compatibility with caches that do not support Variants. 667 5.1.2. Multiple Variants 669 A more complicated request/response pair: 671 GET /murray HTTP/1.1 672 Host: www.example.net 673 Accept-Language: en;q=1.0, fr;q=0.5 674 Accept-Encoding: gzip, br 676 HTTP/1.1 200 OK 677 Content-Type: image/gif 678 Content-Language: en 679 Content-Encoding: br 680 Variants: Accept-Language;en;jp;de 681 Variants: Accept-Encoding;br;gzip 682 Variant-Key: en;br 683 Vary: Accept-Language, Accept-Encoding 684 Transfer-Encoding: chunked 686 Here, the cache knows that there are two axes that the response 687 varies upon; language and encoding. Thus, there are a total of nine 688 possible representations for the resource (including the identity 689 encoding), and the cache needs to consider the selection algorithms 690 for both axes. 692 Upon a subsequent request, if both selection algorithms return a 693 stored representation, it can be served from cache; otherwise, the 694 request will need to be forwarded to origin. 696 5.1.3. Partial Coverage 698 Now, consider the previous example, but where only one of the Vary'd 699 axes (encoding) is listed in Variants: 701 GET /bar HTTP/1.1 702 Host: www.example.net 703 Accept-Language: en;q=1.0, fr;q=0.5 704 Accept-Encoding: gzip, br 705 HTTP/1.1 200 OK 706 Content-Type: image/gif 707 Content-Language: en 708 Content-Encoding: br 709 Variants: Accept-Encoding;br;gzip 710 Variant-Key: br 711 Vary: Accept-Language, Accept-Encoding 712 Transfer-Encoding: chunked 714 Here, the cache will need to calculate a secondary cache key as per 715 [RFC7234], Section 4.1 - but considering only Accept-Language to be 716 in its field-value - and then continue processing Variants for the 717 set of stored responses that the algorithm described there selects. 719 6. Defining Content Negotiation Using Variants 721 To be usable with Variants, proactive content negotiation mechanisms 722 need to be specified to take advantage of it. Specifically, they: 724 o MUST define a request header field that advertises the clients 725 preferences or capabilities, whose field-name SHOULD begin with 726 "Accept-". 728 o MUST define the syntax of an available-value that will occur in 729 Variants and Variant-Key. 731 o MUST define an algorithm for selecting a result. It MUST return a 732 list of available-values that are suitable for the request, in 733 order of preference, given the value of the request header 734 nominated above (or null if the request header is absent) and an 735 available-values list from the Variants header. If the result is 736 an empty list, it implies that the cache cannot satisfy the 737 request. 739 Appendix A fulfils these requirements for some existing proactive 740 content negotiation mechanisms in HTTP. 742 7. IANA Considerations 744 This specification registers the following entry in the Permanent 745 Message Header Field Names registry established by [RFC3864]: 747 o Header field name: Variants 749 o Applicable protocol: http 751 o Status: standard 752 o Author/Change Controller: IETF 754 o Specification document(s): [this document] 756 o Related information: 758 This specification registers the following entry in the Permanent 759 Message Header Field Names registry established by [RFC3864]: 761 o Header field name: Variant-Key 763 o Applicable protocol: http 765 o Status: standard 767 o Author/Change Controller: IETF 769 o Specification document(s): [this document] 771 o Related information: 773 8. Security Considerations 775 If the number or advertised characteristics of the representations 776 available for a resource are considered sensitive, the Variants 777 header by its nature will leak them. 779 Note that the Variants header is not a commitment to make 780 representations of a certain nature available; the runtime behaviour 781 of the server always overrides hints like Variants. 783 9. References 785 9.1. Normative References 787 [I-D.ietf-httpbis-header-structure] 788 Nottingham, M. and P. Kamp, "Structured Headers for HTTP", 789 draft-ietf-httpbis-header-structure-09 (work in progress), 790 December 2018. 792 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 793 Requirement Levels", BCP 14, RFC 2119, 794 DOI 10.17487/RFC2119, March 1997, 795 . 797 [RFC4647] Phillips, A. and M. Davis, "Matching of Language Tags", 798 BCP 47, RFC 4647, DOI 10.17487/RFC4647, September 2006, 799 . 801 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 802 Specifications: ABNF", STD 68, RFC 5234, 803 DOI 10.17487/RFC5234, January 2008, 804 . 806 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 807 Protocol (HTTP/1.1): Message Syntax and Routing", 808 RFC 7230, DOI 10.17487/RFC7230, June 2014, 809 . 811 [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 812 Protocol (HTTP/1.1): Semantics and Content", RFC 7231, 813 DOI 10.17487/RFC7231, June 2014, 814 . 816 [RFC7234] Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, 817 Ed., "Hypertext Transfer Protocol (HTTP/1.1): Caching", 818 RFC 7234, DOI 10.17487/RFC7234, June 2014, 819 . 821 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 822 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 823 May 2017, . 825 9.2. Informative References 827 [I-D.ietf-httpbis-client-hints] 828 Grigorik, I., "HTTP Client Hints", draft-ietf-httpbis- 829 client-hints-06 (work in progress), July 2018. 831 [RFC2295] Holtman, K. and A. Mutz, "Transparent Content Negotiation 832 in HTTP", RFC 2295, DOI 10.17487/RFC2295, March 1998, 833 . 835 [RFC3864] Klyne, G., Nottingham, M., and J. Mogul, "Registration 836 Procedures for Message Header Fields", BCP 90, RFC 3864, 837 DOI 10.17487/RFC3864, September 2004, 838 . 840 9.3. URIs 842 [1] https://lists.w3.org/Archives/Public/ietf-http-wg/ 844 [2] https://httpwg.github.io/ 846 [3] https://github.com/httpwg/http-extensions/labels/variants 848 [4] https://github.com/mnot/variants-toy 850 Appendix A. Variants for Existing Content Negotiation Mechanisms 852 This appendix defines the required information to use existing 853 proactive content negotiation mechanisms (as defined in [RFC7231], 854 Section 5.3) with the Variants header field. 856 A.1. Accept 858 This section defines variant handling for the Accept request header 859 (section 5.3.2 of [RFC7231]). 861 The syntax of an available-value for Accept is: 863 accept-available-value = type "/" subtype 865 To perform content negotiation for Accept given a request-value and 866 available-values: 868 1. Let preferred-available be an empty list. 870 2. Let preferred-types be a list of the types in the request-value 871 (or the empty list if request-value is null), ordered by their 872 weight, highest to lowest, as per Section 5.3.2 of [RFC7231] 873 (omitting any coding with a weight of 0). If a type lacks an 874 explicit weight, an implementation MAY assign one. 876 3. For each preferred-type in preferred-types: 878 1. If any member of available-values matches preferred-type, 879 using the media-range matching mechanism specified in 880 Section 5.3.2 of [RFC7231] (which is case-insensitive), 881 append those members of available-values to preferred- 882 available (preserving the precedence order implied by the 883 media ranges' specificity). 885 4. If preferred-available is empty, append the first member of 886 available-values to preferred-available. This makes the first 887 available-value the default when none of the client's preferences 888 are available. 890 5. Return preferred-available. 892 Note that this algorithm explicitly ignores extension parameters on 893 media types (e.g., "charset"). 895 A.2. Accept-Encoding 897 This section defines variant handling for the Accept-Encoding request 898 header (section 5.3.4 of [RFC7231]). 900 The syntax of an available-value for Accept-Encoding is: 902 accept-encoding-available-value = content-coding / "identity" 904 To perform content negotiation for Accept-Encoding given a request- 905 value and available-values: 907 1. Let preferred-available be an empty list. 909 2. Let preferred-codings be a list of the codings in the request- 910 value (or the empty list if request-value is null), ordered by 911 their weight, highest to lowest, as per Section 5.3.1 of 912 [RFC7231] (omitting any coding with a weight of 0). If a coding 913 lacks an explicit weight, an implementation MAY assign one. 915 3. If "identity" is not a member of preferred-codings, append 916 "identity". 918 4. Append "identity" to available-values. 920 5. For each preferred-coding in preferred-codings: 922 1. If there is a case-insensitive, character-for-character match 923 for preferred-coding in available-values, append that member 924 of available-values to preferred-available. 926 6. Return preferred-available. 928 Note that the unencoded variant needs to have a Variant-Key header 929 field with a value of "identity" (as defined in Section 5.3.4 of 930 [RFC7231]). 932 A.3. Accept-Language 934 This section defines variant handling for the Accept-Language request 935 header (section 5.3.5 of [RFC7231]). 937 The syntax of an available-value for Accept-Language is: 939 accept-encoding-available-value = language-range 941 To perform content negotiation for Accept-Language given a request- 942 value and available-values: 944 1. Let preferred-available be an empty list. 946 2. Let preferred-langs be a list of the language-ranges in the 947 request-value (or the empty list if request-value is null), 948 ordered by their weight, highest to lowest, as per Section 5.3.1 949 of [RFC7231] (omitting any language-range with a weight of 0). 950 If a language-range lacks a weight, an implementation MAY assign 951 one. 953 3. For each preferred-lang in preferred-langs: 955 1. If any member of available-values matches preferred-lang, 956 using either the Basic or Extended Filtering scheme defined 957 in Section 3.3 of [RFC4647], append those members of 958 available-values to preferred-available (preserving their 959 order). 961 4. If preferred-available is empty, append the first member of 962 available-values to preferred-available. This makes the first 963 available-value the default when none of the client's preferences 964 are available. 966 5. Return preferred-available. 968 Acknowledgements 970 This protocol is conceptually similar to, but simpler than, 971 Transparent Content Negotiation [RFC2295]. Thanks to its authors for 972 their inspiration. 974 It is also a generalisation of a Fastly VCL feature designed by 975 Rogier 'DocWilco' Mulhuijzen. 977 Thanks to Hooman Beheshti, Ilya Grigorik, Leif Hedstrom, and Jeffrey 978 Yasskin for their review and input. 980 Author's Address 982 Mark Nottingham 983 Fastly 985 Email: mnot@mnot.net 986 URI: https://www.mnot.net/