idnits 2.17.1 draft-nottingham-variants-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The draft header indicates that this document updates RFC7234, but the abstract doesn't seem to mention this, which it should. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC7234, updated by this document, for RFC5378 checks: 2007-12-21) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (September 28, 2017) is 2396 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7231 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7234 (Obsoleted by RFC 9111) == Outdated reference: A later version (-15) exists of draft-ietf-httpbis-client-hints-04 Summary: 3 errors (**), 0 flaws (~~), 2 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. Nottingham 3 Internet-Draft Fastly 4 Updates: 7234 (if approved) September 28, 2017 5 Intended status: Standards Track 6 Expires: April 1, 2018 8 HTTP Variants 9 draft-nottingham-variants-00 11 Abstract 13 This specification introduces the HTTP "Variants" response header 14 field to communicate what representations are available for a given 15 resource. 17 Note to Readers 19 _RFC EDITOR: please remove this section before publication_ 21 The issues list for this draft can be found at 22 https://github.com/mnot/I-D/labels/variant. 24 The most recent (often, unpublished) draft is at 25 https://mnot.github.io/I-D/variant/. 27 Recent changes are listed at https://github.com/mnot/I-D/commits/gh- 28 pages/variant. 30 See also the draft's current status in the IETF datatracker, at 31 https://datatracker.ietf.org/doc/draft-nottingham-variant/. 33 Status of This Memo 35 This Internet-Draft is submitted in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF). Note that other groups may also distribute 40 working documents as Internet-Drafts. The list of current Internet- 41 Drafts is at http://datatracker.ietf.org/drafts/current/. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on April 1, 2018. 50 Copyright Notice 52 Copyright (c) 2017 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (http://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 68 1.1. Notational Conventions . . . . . . . . . . . . . . . . . 4 69 2. The "Variants" HTTP Header Field . . . . . . . . . . . . . . 4 70 2.1. Defining Content Negotiation Using Variants . . . . . . . 5 71 2.2. Cache Behaviour . . . . . . . . . . . . . . . . . . . . . 6 72 2.2.1. Relationship to Vary . . . . . . . . . . . . . . . . 7 73 2.3. Examples . . . . . . . . . . . . . . . . . . . . . . . . 7 74 2.3.1. Single Variant . . . . . . . . . . . . . . . . . . . 7 75 2.3.2. Multiple Variants . . . . . . . . . . . . . . . . . . 8 76 2.3.3. Partial Coverage . . . . . . . . . . . . . . . . . . 9 77 3. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 78 4. Security Considerations . . . . . . . . . . . . . . . . . . . 9 79 5. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 10 80 6. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 81 6.1. Normative References . . . . . . . . . . . . . . . . . . 10 82 6.2. Informative References . . . . . . . . . . . . . . . . . 11 83 Appendix A. Variants and Defined Content Negotiation Mechanisms 11 84 A.1. Content-Encoding . . . . . . . . . . . . . . . . . . . . 11 85 A.2. Content-Language . . . . . . . . . . . . . . . . . . . . 12 86 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 13 88 1. Introduction 90 HTTP proactive content negotiation ([RFC7231], Section 3.4.1) is 91 starting to be used more widely again. The most widely seen use - 92 determining a response's content-coding - is being joined by renewed 93 interest in negotiation for language and other, newer attributes (for 94 example, see [I-D.ietf-httpbis-client-hints]). 96 Successfully reusing negotiated responses that have been stored in a 97 HTTP cache requires establishment of a secondary cache key 98 ([RFC7234], Section 4.1) using the Vary header ([RFC7231], 99 Section 7.1.4), which identifies the request headers that form the 100 secondary cache key for a given response. 102 HTTP's caching model allows a certain amount of latitude in 103 normalising request header fields to match those stored in the cache, 104 so as to increase the chances of a cache hit while still respecting 105 the semantics of that header. However, this is often inadequate; 106 even with understanding of the headers' semantics to facilitate such 107 normalisation, a cache does not know enough about the possible 108 alternative representations available on the origin server to make an 109 appropriate decision. 111 For example, if a cache has stored the following request/response 112 pair: 114 GET /foo HTTP/1.1 115 Host: www.example.com 116 Accept-Language: en;q=1.0, fr;q=0.5 118 HTTP/1.1 200 OK 119 Content-Type: text/html 120 Content-Language: fr 121 Vary: Accept-Language 122 Transfer-Encoding: chunked 124 [French content] 126 Provided that the cache has full knowledge of the semantics of 127 "Accept-Language" and "Content-Language", it will know that a French 128 representation is available and might be able to infer that an 129 English representation is not available. But, it does not know (for 130 example) whether a Japanese representation is available without 131 making another request, thereby incurring possibly unnecessary 132 latency. 134 This specification introduces the HTTP "Variants" response header 135 field to enumerate the available variant representations on the 136 origin server, to provide clients and caches with enough information 137 to properly satisfy requests - either by selecting a response from 138 cache or by forwarding the request towards the origin. 140 "Variants" is best used when content negotiation takes place over a 141 constrained set of representations; since each variant needs to be 142 listed in the header field, it is ill-suited for open-ended sets of 143 representations. Likewise, it works best for content negotiation 144 over header fields whose semantics are well-understood, since it 145 requires a selection algorithm to be specified ahead of time. 147 1.1. Notational Conventions 149 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 150 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 151 "OPTIONAL" in this document are to be interpreted as described in BCP 152 14 [RFC2119] [RFC8174] when, and only when, they appear in all 153 capitals, as shown here. 155 This specification uses the Augmented Backus-Naur Form (ABNF) 156 notation of [RFC5234] with a list extension, defined in Section 7 of 157 [RFC7230], that allows for compact definition of comma-separated 158 lists using a '#' operator (similar to how the '*' operator indicates 159 repetition). 161 Additionally, it uses the "field-name", "OWS" and "token" rules from 162 [RFC7230]. 164 2. The "Variants" HTTP Header Field 166 The "Variant" HTTP response header field is used to indicate what 167 other representations are available for a given resource at the time 168 that the response is produced. 170 Variants = 1#variant 171 variant = field-name *( OWS ";" OWS available-value ) 172 available-value = token 174 Each "variant" indicates a response header field that carries a value 175 that clients might proactively negotiate for; each parameter on it 176 indicates a value for which there is an available representation on 177 the origin server. 179 So, given this example header field: 181 Variants: Content-Encoding;gzip 183 a recipient can infer that the only content-coding available for that 184 resource is "gzip" (along with the "identity" non-encoding; see 185 {{content-encoding}). 187 Given: 189 Variants: content-encoding 191 a recipient can infer that no content-codings are supported. Note 192 that as always with header field names, it is case-insensitive. 194 A more complex example: 196 Variants: Content-Encoding;gzip;brotli, Content-Language;en ;fr 198 Here, recipients can infer that two Content-Encodings are available, 199 as well as two content languages. Note that, as with all HTTP header 200 fields that use the "#" list rule (see [RFC7230], Section 7), they 201 might occur in the same header field or separately, like this: 203 Variants: Content-Encoding;gzip;brotli 204 Variants: Content-Language;en ;fr 206 The ordering of available-values after the field-name is significant, 207 as it might be used by the header's algorithm for selecting a 208 response. 210 Senders SHOULD consistently send "Variant" header fields on all 211 cacheable (as per [RFC7234], Section 3) responses for a resource, 212 since its absence will trigger caches to fall back to "Vary" 213 processing. 215 Likewise, servers MUST send the "Content-*" response headers 216 nominated by "Variants" when sending that header. 218 2.1. Defining Content Negotiation Using Variants 220 To be usable with Variants, proactive content negotiation mechanisms 221 need to be specified to take advantage of it. Specifically, they: 223 o MUST define a request header field that advertises the clients 224 preferences or capabilities, whose field-name SHOULD begin with 225 "Accept-". 227 o MUST define a response header field that indicates the result of 228 selection, whose field-name SHOULD begin with "Content-" and whose 229 field-value SHOULD be a token. 231 o MUST define an algorithm for selecting a result. It MUST return 232 an ordered list of selected responses, given the incoming request, 233 a list of selected responses, and the list of available values 234 from "Variants". If the result is an empty list, it implies that 235 the cache does not contain an appropriate response. 237 Appendix A fulfils these requirements for some existing proactive 238 content negotiation mechanisms in HTTP. 240 Note that unlike Vary, Variants does not use stored request headers 241 to help select a response; this is why defining a response header to 242 aid identification and selection is required. 244 2.2. Cache Behaviour 246 Caches that implement the "Variants" header field and the relevant 247 semantics of the field-name it contains can use that knowledge to 248 either select an appropriate stored representation, or forward the 249 request if no appropriate representation is stored. 251 They do so by running this algorithm (or its functional equivalent) 252 upon receiving a request, "incoming-request": 254 1. Let "selected-responses" be a list of the stored responses 255 suitable for reuse as defined in [RFC7234] Section 4, excepting 256 the requirement to calculate a secondary cache key. 258 2. Order "selected-responses" by the "Date" header field, most 259 recent to least recent. 261 3. If the freshest (as per [RFC7234], Section 4.2) has one or more 262 "Variants" header field(s): 264 1. Select one member of "selected-responses" and let its 265 "Variants" header field-value(s) be "Variants". This SHOULD 266 be the most recent response, but MAY be from an older one as 267 long as it is still fresh. 269 2. For each "variant" in "Variants": 271 1. If the "field-name" corresponds to the response header 272 field identified by a content negotiation mechanism that 273 the implementation supports: 275 1. Let "available-values" be a list containing all 276 "available-value" for the "variant". 278 2. Let "selected-responses" be the result of running the 279 algorithm defined by the content negotiation 280 mechanism with "incoming-request", "selected- 281 responses" and "available-values". 283 3. For the purposes of selecting a response, ignore the 284 content negotiation's identified request header 285 field-name in the "Vary" header field of each member 286 of "selected-responses", if present. 288 4. Process any member of "selected-responses" that has a "Vary" 289 response header field whose field-value still contains one or 290 more "field-name"s, removing that members if it does not match 291 (as per [RFC7234], Section 4.1). 293 5. Return the first member of "selected-responses". If "selected- 294 responses" is empty, return "null". 296 This algorithm will either return the appropriate stored response to 297 use, or "null" if the cache needs to forward the request towards the 298 origin server. 300 2.2.1. Relationship to Vary 302 Caches that fully implement this specification MUST ignore request 303 header-fields in the "Vary" header for the purposes of secondary 304 cache key calculation ([RFC7234], Section 4.1) when their semantics 305 are understood, implemented as per this specification, and their 306 corresponding response header field is listed in "Variants". 308 Request header fields listed in "Vary" that are not implemented in 309 terms of this specification or not present in the "Variants" field 310 SHOULD still form part of the secondary cache key. 312 The algorithm in Section 2.2 implements these requirements. 314 2.3. Examples 316 2.3.1. Single Variant 318 Given a request/response pair: 320 GET /foo HTTP/1.1 321 Host: www.example.com 322 Accept-Language: en;q=1.0, fr;q=0.5 324 HTTP/1.1 200 OK 325 Content-Type: image/gif 326 Content-Language: en 327 Cache-Control: max-age=3600 328 Variants: Content-Language;en;de 329 Vary: Accept-Language 330 Transfer-Encoding: chunked 332 Upon receipt of this response, the cache knows that two 333 representations of this resource are available, one with a "Content- 334 Language" of "en", and another whose "Content-Language" is "de". 336 Subsequent requests (while this response is fresh) will cause the 337 cache to either reuse this response or forward the request, depending 338 on what the selection algorithm "Accept-Language" and "Content- 339 Language" determines. 341 So, a request with "en" in "Accept-Language" is received and its 342 q-value indicates that it is acceptable, the stored response is used. 343 A request that indicates that "de" is acceptable will be forwarded to 344 the origin, thereby populating the cache. A cache receiving a 345 request that indicates both languages are acceptable will use the 346 q-value to make a determination of what response to return. 348 A cache receiving a request that does not list either language as 349 acceptable (or does not contain an Accept-Language at all) will 350 return the "en" representation (possibly fetching it from the 351 origin), since it is listed first in the "Variants" list. 353 Note that "Accept-Language" is listed in Vary, to assure backwards- 354 compatibility with caches that do not support "Variants". 356 Also, note that is is the response header which is listed in 357 Variants, not the request header (the opposite of Vary). 359 2.3.2. Multiple Variants 361 A more complicated request/response pair: 363 GET /bar HTTP/1.1 364 Host: www.example.net 365 Accept-Language: en;q=1.0, fr;q=0.5 366 Accept-Encoding: gzip, br 368 HTTP/1.1 200 OK 369 Content-Type: image/gif 370 Content-Language: en 371 Content-Encoding: br 372 Variants: Content-Language;en;jp;de 373 Variants: Content-Encoding;br;gzip 374 Vary: Accept-Language, Accept-Encoding 375 Transfer-Encoding: chunked 377 Here, the cache knows that there are two axes that the response 378 varies upon; "Content-Language" and "Content-Encoding". Thus, there 379 are a total of six possible representations for the resource, and the 380 cache needs to consider the selection algorithms for both axes. 382 Upon a subsequent request, if both selection algorithms return a 383 stored representation, it can be served from cache; otherwise, the 384 request will need to be forwarded to origin. 386 2.3.3. Partial Coverage 388 Now, consider the previous example, but where only one of the varied 389 axes is listed in "Variants": 391 GET /bar HTTP/1.1 392 Host: www.example.net 393 Accept-Language: en;q=1.0, fr;q=0.5 394 Accept-Encoding: gzip, br 396 HTTP/1.1 200 OK 397 Content-Type: image/gif 398 Content-Language: en 399 Content-Encoding: br 400 Variants: Content-Encoding;br;gzip 401 Vary: Accept-Language, Accept-Encoding 402 Transfer-Encoding: chunked 404 Here, the cache will need to calculate a secondary cache key as per 405 [RFC7234], Section 4.1 - but considering only "Accept-Language" to be 406 in its field-value - and then continue processing "Variants" for the 407 set of stored responses that the algorithm described there selects. 409 3. IANA Considerations 411 This specification registers one value in the Permanent Message 412 Header Field Names registry established by [RFC3864]: 414 o Header field name: Variants 416 o Applicable protocol: http 418 o Status: standard 420 o Author/Change Controller: IETF 422 o Specification document(s): [this document] 424 o Related information: 426 4. Security Considerations 428 If the number or advertised characteristics of the representations 429 available for a resource are considered sensitive, the "Variants" 430 header by its nature will leak them. 432 Note that the "Variants" header is not a commitment to make 433 representations of a certain nature available; the runtime behaviour 434 of the server always overrides hints like "Variants". 436 5. Acknowledgments 438 This protocol is conceptually similar to, but simpler than, 439 Transparent Content Negotiation [RFC2295]. Thanks to its authors for 440 their inspiration. 442 It is also a generalisation of a Fastly VCL feature designed by 443 Rogier 'DocWilco' Mulhuijzen. 445 Thanks to Hooman Beheshti for his review and input. 447 6. References 449 6.1. Normative References 451 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 452 Requirement Levels", BCP 14, RFC 2119, 453 DOI 10.17487/RFC2119, March 1997, . 456 [RFC4647] Phillips, A. and M. Davis, "Matching of Language Tags", 457 BCP 47, RFC 4647, DOI 10.17487/RFC4647, September 2006, 458 . 460 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 461 Specifications: ABNF", STD 68, RFC 5234, 462 DOI 10.17487/RFC5234, January 2008, . 465 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 466 Protocol (HTTP/1.1): Message Syntax and Routing", 467 RFC 7230, DOI 10.17487/RFC7230, June 2014, 468 . 470 [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 471 Protocol (HTTP/1.1): Semantics and Content", RFC 7231, 472 DOI 10.17487/RFC7231, June 2014, . 475 [RFC7234] Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, 476 Ed., "Hypertext Transfer Protocol (HTTP/1.1): Caching", 477 RFC 7234, DOI 10.17487/RFC7234, June 2014, 478 . 480 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 481 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 482 May 2017, . 484 6.2. Informative References 486 [I-D.ietf-httpbis-client-hints] 487 Grigorik, I., "HTTP Client Hints", draft-ietf-httpbis- 488 client-hints-04 (work in progress), April 2017. 490 [RFC2295] Holtman, K. and A. Mutz, "Transparent Content Negotiation 491 in HTTP", RFC 2295, DOI 10.17487/RFC2295, March 1998, 492 . 494 [RFC3864] Klyne, G., Nottingham, M., and J. Mogul, "Registration 495 Procedures for Message Header Fields", BCP 90, RFC 3864, 496 DOI 10.17487/RFC3864, September 2004, . 499 Appendix A. Variants and Defined Content Negotiation Mechanisms 501 This appendix defines the required information to use existing 502 proactive content negotiation mechanisms (as defined in [RFC7231], 503 Section 5.3) with the "Variants" header field. 505 A.1. Content-Encoding 507 When negotiating for the "Content-Encoding" response header field's 508 value, the applicable request header field is "Accept-Encoding", as 509 per [RFC7231] Section 5.3.4. 511 To perform content negotiation for Content-Encoding given an 512 "incoming-request", "stored-responses" and "available-values": 514 1. Let "preferred-codings" be a list of the "coding"s in the 515 "Accept-Encoding" header field of "incoming-request", ordered by 516 their "weight", highest to lowest. If "Accept-Encoding" is not 517 present or empty, "preferred-codings" will be empty. 519 2. If "identity" is not a member of "preferred-codings", append 520 "identity" to "preferred-codings" with a "weight" of 0.001. 522 3. Remove any member of "preferred-codings" whose "weight" is 0. 524 4. Append "identity" to "available-values". 526 5. Remove any member of "available-values" not present in 527 "preferred-codings", comparing in a case-insensitive fashion. 529 6. Let "filtered-responses" be an empty list. 531 7. For each "available-value" of "available-values": 533 1. If there is a member of "stored-responses" whose "Content- 534 Encoding" field-value has "content-coding"s ([RFC7231], 535 Section 3.1.2.2) that all match members of "available-value" 536 in a case-insensitive fashion, append that stored response to 537 "filtered-responses". 539 8. If there is a member of "stored-responses" that does not have a 540 "Content-Encoding" header field, append that stored response to 541 "filtered-responses". 543 9. Return "filtered-responses". 545 This algorithm selects the stored response(s) in order of preference 546 by the client; if none are stored in cache, the request will be 547 forwarded towards the origin. It defaults to the "identity" non- 548 encoding. 550 Implementations MAY remove members of "filtered-responses" based upon 551 their "weight" or other criteria before returning. For example, they 552 might wish to return an empty list when the client's most-preferred 553 available response is not stored, so as to populate the cache as well 554 as honour the client's preferences. 556 A.2. Content-Language 558 When negotiating for the "Content-Language" response header field's 559 value, the applicable request header field is "Accept-Language", as 560 per [RFC7231] Section 5.3.5. 562 To perform content negotiation for Content-Language given an 563 "incoming-request", "stored-responses" and "available-values": 565 1. Let "preferred-langs" be a list of the "language-range"s in the 566 "Accept-Language" header field ([RFC7231], Section 5.3.5) of 567 "incoming-request", ordered by their "weight", highest to lowest. 569 2. If "preferred-langs" is empty, append "*" with a "weight" of 570 0.001. 572 3. Remove any member of "preferred-langs" whose "weight" is 0. 574 4. Filter "available-values" using "preferred-langs" with either the 575 Basic Filtering scheme defined in [RFC4647] Section 3.3.1, or the 576 Lookup scheme defined in Section 3.4 of that document. Use the 577 first member of "available-values" as the default. 579 5. Let "filtered-responses" be an empty list. 581 6. For each "available-value" of "available-values": 583 1. If there is a member of "stored-responses" whose "Content- 584 Language" field-value has a "language-tag" ([RFC7231], 585 Section 3.1.3.2) that matches "available-value" in a case- 586 insensitive fashion, append that stored response to 587 "filtered-responses". 589 7. Return "filtered-responses". 591 This algorithm selects the available response(s) (according to 592 "Variants") in order of preference by the client; if none are stored 593 in cache, the request will be forwarded towards the origin. If no 594 preferred language can be selected, the first "available-value" will 595 be used as the default. 597 Implementations MAY remove members of "filtered-responses" based upon 598 their "weight" or other criteria before returning. For example, they 599 might wish to return an empty list when the client's most-preferred 600 available response is not stored, so as to populate the cache as well 601 as honour the client's preferences. 603 Author's Address 605 Mark Nottingham 606 Fastly 608 Email: mnot@mnot.net 609 URI: https://www.mnot.net/