< draft-nottingham-variants-01.txt   draft-nottingham-variants-02.txt >
Network Working Group M. Nottingham Network Working Group M. Nottingham
Internet-Draft Fastly Internet-Draft Fastly
Updates: 7234 (if approved) October 30, 2017 Updates: 7234 (if approved) February 13, 2018
Intended status: Standards Track Intended status: Standards Track
Expires: May 3, 2018 Expires: August 17, 2018
HTTP Representation Variants HTTP Representation Variants
draft-nottingham-variants-01 draft-nottingham-variants-02
Abstract Abstract
This specification introduces the HTTP "Variants" response header This specification introduces an alternative way to communicate a
field to communicate what representations are available for a given secondary cache key for a HTTP resource, using the HTTP "Variants"
resource, and the companion "Variant-Key" response header field to and "Variant-Key" response header fields. Its aim is to make HTTP
indicate which representation was selected. It is an augmentation of proactive content negotiation more cache-friendly.
the "Vary" mechanism in HTTP caching.
Note to Readers Note to Readers
_RFC EDITOR: please remove this section before publication_ _RFC EDITOR: please remove this section before publication_
The issues list for this draft can be found at The issues list for this draft can be found at
https://github.com/mnot/I-D/labels/variants [1]. https://github.com/mnot/I-D/labels/variants [1].
The most recent (often, unpublished) draft is at The most recent (often, unpublished) draft is at
https://mnot.github.io/I-D/variants/ [2]. https://mnot.github.io/I-D/variants/ [2].
Recent changes are listed at https://github.com/mnot/I-D/commits/gh- Recent changes are listed at https://github.com/mnot/I-D/commits/gh-
pages/variants [3]. pages/variants [3].
See also the draft's current status in the IETF datatracker, at See also the draft's current status in the IETF datatracker, at
https://datatracker.ietf.org/doc/draft-nottingham-variants/ [4]. https://datatracker.ietf.org/doc/draft-nottingham-variants/ [4].
There is a prototype implementation of the algorithms herein at
https://github.com/mnot/variants-toy [5].
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on May 3, 2018.
This Internet-Draft will expire on August 17, 2018.
Copyright Notice Copyright Notice
Copyright (c) 2017 IETF Trust and the persons identified as the Copyright (c) 2018 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Notational Conventions . . . . . . . . . . . . . . . . . 4 1.1. Notational Conventions . . . . . . . . . . . . . . . . . 4
2. The "Variants" HTTP Header Field . . . . . . . . . . . . . . 4 2. The "Variants" HTTP Header Field . . . . . . . . . . . . . . 5
2.1. Relationship to Vary . . . . . . . . . . . . . . . . . . 6 2.1. Relationship to Vary . . . . . . . . . . . . . . . . . . 6
3. The "Variant-Key" HTTP Header Field . . . . . . . . . . . . . 6 3. The "Variant-Key" HTTP Header Field . . . . . . . . . . . . . 7
4. Defining Content Negotiation Using Variants . . . . . . . . . 7 3.1. Generating a Normalised Variant-Key . . . . . . . . . . . 7
5. Cache Behaviour . . . . . . . . . . . . . . . . . . . . . . . 7 4. Cache Behaviour . . . . . . . . . . . . . . . . . . . . . . . 8
5.1. Find Available Keys . . . . . . . . . . . . . . . . . . . 9 4.1. Find Available Keys . . . . . . . . . . . . . . . . . . . 9
5.2. Example of Cache Behaviour . . . . . . . . . . . . . . . 9 4.2. Check Vary . . . . . . . . . . . . . . . . . . . . . . . 10
6. Example Headers . . . . . . . . . . . . . . . . . . . . . . . 10 4.3. Example of Cache Behaviour . . . . . . . . . . . . . . . 10
6.1. Single Variant . . . . . . . . . . . . . . . . . . . . . 10 5. Origin Server Behaviour . . . . . . . . . . . . . . . . . . . 11
6.2. Multiple Variants . . . . . . . . . . . . . . . . . . . . 11 5.1. Examples . . . . . . . . . . . . . . . . . . . . . . . . 12
6.3. Partial Coverage . . . . . . . . . . . . . . . . . . . . 11 5.1.1. Single Variant . . . . . . . . . . . . . . . . . . . 12
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 5.1.2. Multiple Variants . . . . . . . . . . . . . . . . . . 13
8. Security Considerations . . . . . . . . . . . . . . . . . . . 13 5.1.3. Partial Coverage . . . . . . . . . . . . . . . . . . 13
9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 13 6. Defining Content Negotiation Using Variants . . . . . . . . . 14
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 13 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14
10.1. Normative References . . . . . . . . . . . . . . . . . . 13 8. Security Considerations . . . . . . . . . . . . . . . . . . . 15
10.2. Informative References . . . . . . . . . . . . . . . . . 14 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 15
10.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 14 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 15
Appendix A. Variants for Existing Content Negotiation Mechanisms 14 10.1. Normative References . . . . . . . . . . . . . . . . . . 15
A.1. Accept-Encoding . . . . . . . . . . . . . . . . . . . . . 14 10.2. Informative References . . . . . . . . . . . . . . . . . 16
A.2. Accept-Language . . . . . . . . . . . . . . . . . . . . . 15 10.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 15 Appendix A. Variants for Existing Content Negotiation Mechanisms 17
A.1. Accept . . . . . . . . . . . . . . . . . . . . . . . . . 17
A.2. Accept-Encoding . . . . . . . . . . . . . . . . . . . . . 17
A.3. Accept-Language . . . . . . . . . . . . . . . . . . . . . 18
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 19
1. Introduction 1. Introduction
HTTP proactive content negotiation ([RFC7231], Section 3.4.1) is HTTP proactive content negotiation ([RFC7231], Section 3.4.1) is
starting to be used more widely again. The most widely seen use - seeing renewed interest, both for existing request headers like
determining a response's content-coding - is being joined by renewed Content-Language and for newer ones (for example, see
interest in negotiation for language and other, newer attributes (for [I-D.ietf-httpbis-client-hints]).
example, see [I-D.ietf-httpbis-client-hints]).
Successfully reusing negotiated responses that have been stored in a Successfully reusing negotiated responses that have been stored in a
HTTP cache requires establishment of a secondary cache key HTTP cache requires establishment of a secondary cache key
([RFC7234], Section 4.1) using the Vary header ([RFC7231], ([RFC7234], Section 4.1). Currently, the Vary header ([RFC7231],
Section 7.1.4), which identifies the request headers that form the Section 7.1.4) does this by nominating a set of request headers.
secondary cache key for a given response.
HTTP's caching model allows a certain amount of latitude in HTTP's caching model allows a certain amount of latitude in
normalising request header fields identified by Vary to match those normalising those request header field values, so as to increase the
stored in the cache, so as to increase the chances of a cache hit chances of a cache hit while still respecting the semantics of that
while still respecting the semantics of that header. However, this header. However, normalisation is not formally defined, leading to
is often inadequate; even with understanding of the headers' divergence in cache behaviours.
semantics to facilitate such normalisation, a cache does not know
enough about the possible alternative representations available on Even when the headers' semantics are understood, a cache does not
the origin server to make an appropriate decision. know enough about the possible alternative representations available
on the origin server to make an appropriate decision.
For example, if a cache has stored the following request/response For example, if a cache has stored the following request/response
pair: pair:
GET /foo HTTP/1.1 GET /foo HTTP/1.1
Host: www.example.com Host: www.example.com
Accept-Language: en;q=1.0, fr;q=0.5 Accept-Language: en;q=1.0, fr;q=0.5
HTTP/1.1 200 OK HTTP/1.1 200 OK
Content-Type: text/html Content-Type: text/html
skipping to change at page 3, line 48 skipping to change at page 3, line 49
Vary: Accept-Language Vary: Accept-Language
Transfer-Encoding: chunked Transfer-Encoding: chunked
[French content] [French content]
Provided that the cache has full knowledge of the semantics of Provided that the cache has full knowledge of the semantics of
Accept-Language and Content-Language, it will know that a French Accept-Language and Content-Language, it will know that a French
representation is available and might be able to infer that an representation is available and might be able to infer that an
English representation is not available. But, it does not know (for English representation is not available. But, it does not know (for
example) whether a Japanese representation is available without example) whether a Japanese representation is available without
making another request, thereby incurring possibly unnecessary making another request, incurring possibly unnecessary latency.
latency.
This specification introduces the HTTP Variants response header field This specification introduces the HTTP Variants response header field
(Section 2) to enumerate the available variant representations on the (Section 2) to enumerate the available variant representations on the
origin server, to provide clients and caches with enough information origin server, to provide clients and caches with enough information
to properly satisfy requests - either by selecting a response from to properly satisfy requests - either by selecting a response from
cache or by forwarding the request towards the origin - by following cache or by forwarding the request towards the origin - by following
an algorithm defined in Section 5. the algorithm defined in Section 4.
Its companion the Variant-Key response header field (Section 3) Its companion the Variant-Key response header field (Section 3)
indicates which representation was selected, so that it can be indicates which representation was selected, so that it can be
reliably reused in the future. reliably reused in the future. When this specification is in use,
the example above might become:
This mechanism requires that proactive content negotiation mechanisms GET /foo HTTP/1.1
define how they use it; see Section 4. It is best suited for Host: www.example.com
negotiation over request headers that are well-understood. It also Accept-Language: en;q=1.0, fr;q=0.5
works best when content negotiation takes place over a constrained
set of representations; since each variant needs to be listed in the HTTP/1.1 200 OK
header field, it is ill-suited for open-ended sets of Content-Type: text/html
Content-Language: fr
Vary: Accept-Language
Variants: Accept-Language;fr;de;en;jp
Variant-Key: fr
Transfer-Encoding: chunked
[French content]
Proactive content negotiation mechanisms that wish to be used with
Variants need to define how to do so explicitly; see Section 6. As a
result, it is best suited for negotiation over request headers that
are well-understood.
Variants also works best when content negotiation takes place over a
constrained set of representations; since each variant needs to be
listed in the header field, it is ill-suited for open-ended sets of
representations. representations.
It can be seen as a simpler version of the Alternates header field Variants can be seen as a simpler version of the Alternates header
introduced by [RFC2295]; unlike that mechanism, Variants does not field introduced by [RFC2295]; unlike that mechanism, Variants does
require specification of each combination of attributes, and does not not require specification of each combination of attributes, and does
assume that each combination has a unique URL. not assume that each combination has a unique URL.
1.1. Notational Conventions 1.1. Notational Conventions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP "OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all 14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here. capitals, as shown here.
This specification uses the Augmented Backus-Naur Form (ABNF) This specification uses the Augmented Backus-Naur Form (ABNF)
skipping to change at page 5, line 20 skipping to change at page 5, line 36
value that clients might proactively negotiate for; each parameter on value that clients might proactively negotiate for; each parameter on
it indicates a value for which there is an available representation it indicates a value for which there is an available representation
on the origin server. on the origin server.
So, given this example header field: So, given this example header field:
Variants: Accept-Encoding;gzip Variants: Accept-Encoding;gzip
a recipient can infer that the only content-coding available for that a recipient can infer that the only content-coding available for that
resource is "gzip" (along with the "identity" non-encoding; see resource is "gzip" (along with the "identity" non-encoding; see
Appendix A.1). Appendix A.2).
Given: Given:
Variants: accept-encoding Variants: accept-encoding
a recipient can infer that no content-codings (beyond identity) are a recipient can infer that no content-codings (beyond identity) are
supported. Note that as always, field-name is case-insensitive. supported. Note that as always, field-name is case-insensitive.
A more complex example: A more complex example:
skipping to change at page 5, line 36 skipping to change at page 6, line 4
a recipient can infer that no content-codings (beyond identity) are a recipient can infer that no content-codings (beyond identity) are
supported. Note that as always, field-name is case-insensitive. supported. Note that as always, field-name is case-insensitive.
A more complex example: A more complex example:
Variants: Accept-Encoding;gzip;br, Accept-Language;en ;fr Variants: Accept-Encoding;gzip;br, Accept-Language;en ;fr
Here, recipients can infer that two content-codings in addition to Here, recipients can infer that two content-codings in addition to
"identity" are available, as well as two content languages. Note "identity" are available, as well as two content languages. Note
that, as with all HTTP header fields that use the "#" list rule (see that, as with all HTTP header fields that use the "#" list rule (see
[RFC7230], Section 7), they might occur in the same header field or [RFC7230], Section 7), they might occur in the same header field or
separately, like this: separately, like this:
Variants: Accept-Encoding;gzip;brotli Variants: Accept-Encoding;gzip;brotli
Variants: Accept-Language;en ;fr Variants: Accept-Language;en ;fr
The ordering of available-values after the field-name is significant, The ordering of available-values after the field-name is significant,
as it might be used by the header's algorithm for selecting a as it might be used by the header's algorithm for selecting a
response (see Appendix A.1 for an example of this). response (in this example, the first language is the default; see
Appendix A.3).
The ordering of the request header fields themselves indicates The ordering of the request header fields themselves indicates
descending application of preferences; for example, in the headers descending application of preferences; in the example above, a cache
above, a cache will serve gzip'd content regardless of language if it that has all of the possible permutations stored will honour the
is available. client's preferences for Accept-Encoding before honouring Accept-
Language.
Origin servers SHOULD consistently send Variant header fields on all Origin servers SHOULD consistently send Variant header fields on all
cacheable (as per [RFC7234], Section 3) responses for a resource, cacheable (as per [RFC7234], Section 3) responses for a resource,
since its absence will trigger caches to fall back to Vary since its absence will trigger caches to fall back to Vary
processing. processing.
Likewise, servers MUST send the Variant-Key response header field Likewise, servers MUST send the Variant-Key response header field
when sending Variants. when sending Variants, since its absence means that the stored
response will not be reused when this specification is implemented.
2.1. Relationship to Vary 2.1. Relationship to Vary
Caches that fully implement this specification SHOULD ignore request Caches that implement this specification SHOULD ignore request header
header fields in the "Vary" header for the purposes of secondary fields in the Vary header for the purposes of secondary cache key
cache key calculation ([RFC7234], Section 4.1) when their semantics calculation ([RFC7234], Section 4.1) when their semantics are
are implemented as per this specification and their corresponding implemented as per this specification and their corresponding
response header field is listed in "Variants". response header field is listed in Variants.
If any member of the Vary header does not have a corresponding If any member of the Vary header does not have a corresponding
variant that is understood by the implementation, it is still subject variant that is understood by the implementation, it is still subject
to the requirements there. to the requirements there.
See Section 5.1.3 for an example.
In practice, implementation of Vary varies considerably. As a
result, cache efficiency might drop considerably when Variants does
not contain all of the headers referenced by Vary, because some
implementations might choose to disable Variants processing when this
is the case.
3. The "Variant-Key" HTTP Header Field 3. The "Variant-Key" HTTP Header Field
The Variant-Key HTTP response header field is used to indicate the The Variant-Key HTTP response header field is used to indicate the
value(s) from the Variants header field that identify the value(s) from the Variants header field that identify the
representation it occurs within. representation it occurs within.
Variant-Key = 1#available-value Variant-Key = 1#available-value
Each value indicates the selected available-value, in the same order Each value indicates the selected available-value, in the same order
as the variants listed in the Variants header field. as the variants listed in the Variants header field.
Therefore, Variant-Key MUST be the same length (in comma-separated Therefore, Variant-Key MUST be the same length (in comma-separated
members) as Variants, and each member MUST correspond in position to members) as Variants, and each member MUST correspond in position to
its companion in Variants. its companion in Variants.
For example: For example:
Variants: Content-Encoding;gzip;br, Content-Language;en ;fr Variants: Content-Encoding;gzip;br, Content-Language;en ;fr
Variant-Key: gzip, fr Variant-Key: gzip, fr
This header pair indicates that the representation is used for This header pair indicates that the representation has a "gzip"
responses that have a "gzip" content-coding and "fr" content- content-coding and "fr" content-language.
language.
Note that the contents of Variant-Key are only used to indicate what Note that Variant-Key is only used to indicate what request
request attributes are identified with the response containing it; attributes are associated with the response containing it; this is
this is different from headers like Content-Encoding, which indicate different from headers like Content-Encoding, which indicate
attributes of the response. In the example above, it might be that a attributes of the response itself. In the example above, it might be
gzip'd version of the French content is not available, in which case that a gzip'd version of the French content is not available, in
it will not include "Content-Encoding: gzip", but still have "gzip" which case the response will include:
in Variant-Key.
4. Defining Content Negotiation Using Variants Variant-Key: gzip, fr
To be usable with Variants, proactive content negotiation mechanisms even though Content-Encoding does not contain "gzip".
need to be specified to take advantage of it. Specifically, they:
o MUST define a request header field that advertises the clients 3.1. Generating a Normalised Variant-Key
preferences or capabilities, whose field-name SHOULD begin with
"Accept-".
o MUST define the syntax of available-values that will occur in This algorithm generates a normalised string for Variant-Key,
Variants and Variant-Key. suitable for comparison with values generated by Section 4.
o MUST define an algorithm for selecting a result. It MUST return a Given stored-headers, a set of headers from a stored response, a
list of available-values that are suitable for the request, in normalised variant-key for that message can be generated by:
order of preference, given the value of the request header
nominated above and an available-values list from the Variants
header. If the result is an empty list, it implies that the cache
cannot satisfy the request.
Appendix A fulfils these requirements for some existing proactive 1. Let variant-key-header be a string, the result of selecting all
content negotiation mechanisms in HTTP. field-values of stored-headers whose field-name is "Variant-Key"
and joining them with a comma (",").
5. Cache Behaviour 2. Remove all whitespace from variant-key-header.
3. Return variant-key-header.
4. Cache Behaviour
Caches that implement the Variants header field and the relevant Caches that implement the Variants header field and the relevant
semantics of the field-name it contains can use that knowledge to semantics of the field-name it contains can use that knowledge to
either select an appropriate stored representation, or forward the either select an appropriate stored representation, or forward the
request if no appropriate representation is stored. request if no appropriate representation is stored.
They do so by running this algorithm (or its functional equivalent) They do so by running this algorithm (or its functional equivalent)
upon receiving a request, incoming-request: upon receiving a request:
1. Let selected-responses be a list of the stored responses suitable Given incoming-request, a mapping of field-names to lists of field
for reuse as defined in [RFC7234] Section 4, excepting the values, and stored-responses, a list of stored responses suitable for
requirement to calculate a secondary cache key. reuse as defined in [RFC7234] Section 4, excepting the requirement to
calculate a secondary cache key:
2. Order selected-responses by the "Date" header field, most recent 1. If stored-responses is empty, return an empty list.
to least recent.
3. If the freshest (as per [RFC7234], Section 4.2) has one or more 2. Order stored-responses by the "Date" header field, most recent to
"Variants" header field(s): least recent.
1. Select one member of selected_responses and let its 3. Let sorted-variants be an empty list.
"Variants" header field-value(s) be variants-header. This
SHOULD be the most recent response, but MAY be from an older
one as long as it is still fresh.
2. Let sorted-variants be an empty list. 4. If the freshest member of stored-responses (as per [RFC7234],
Section 4.2) has one or more "Variants" header field(s):
3. For each variant in variants-header: 1. Select one member of stored-responses and let its "Variants"
header field-value(s) be variants-header. This SHOULD be the
most recent response, but MAY be from an older one as long as
it is still fresh.
1. If variant's field-name corresponds to the response 2. For each variant in variants-header:
header field identified by a content negotiation
mechanism that the implementation supports:
1. Let request-value be the field-value of the request 1. If variant's field-name corresponds to the request header
header field(s) identified by the content negotiation field identified by a content negotiation mechanism that
mechanism. the implementation supports:
1. Let request-value be the field-value(s) associated
with field-name in incoming-request.
2. Let available-values be a list containing all 2. Let available-values be a list containing all
available-value for the variant. available-value for variant.
3. Let sorted-values be the result of running the 3. Let sorted-values be the result of running the
algorithm defined by the content negotiation algorithm defined by the content negotiation
mechanism with request-value and available-values. mechanism with request-value and available-values.
4. Append sorted-values to sorted-variants. 4. Append sorted-values to sorted-variants.
At this point, sorted-variants will be a list of lists, each At this point, sorted-variants will be a list of lists, each
member of the top-level list corresponding to a variant-item member of the top-level list corresponding to a variant-item
in the Variants header field-value, containing zero or more in the Variants header field-value, containing zero or more
items indicating available-values that are acceptable to the items indicating available-values that are acceptable to the
client, in order of preference, greatest to least. client, in order of preference, greatest to least.
4. If any member of sorted-variants is an empty list, stop 5. Return result of running Find Available Keys (Section 4.1) on
processing and forward the request towards the origin, since sorted-variants, an empty string and an empty list.
an acceptable response is not stored in the cache.
5. Let sorted-keys be the result of running Find Available Keys
(Section 5.1) on sorted-variants and two empty lists.
This will result in a list of lists, where each member of the top-
level list indicates, in preference order, a key for an acceptable
response to the request.
A Cache MAY satisfy the request with any response whose Variant-Key
header corresponds to a member of sorted-keys; when doing so, it
SHOULD use the most preferred available response.
See also Section 2.1 regarding handling of Vary. This returns a list of strings suitable for comparing to normalised
Variant-Keys (Section 3.1) that represent possible responses on the
server that can be used to satisfy the request, in preference order,
provided that their secondary cache key (after removing the headers
covered by Variants) matches. Section 4.2 illustrates one way to do
this.
5.1. Find Available Keys 4.1. Find Available Keys
Given sorted-variants, a list of lists, and key-stub, a list Given sorted-variants, a list of lists, and key-stub, a string
representing a partial key, and possible-keys, a list: representing a partial key, and possible-keys, a list:
1. Let sorted-values be the first member of sorted-variants. 1. Let sorted-values be the first member of sorted-variants.
2. For each sorted-value in sorted-values: 2. For each sorted-value in sorted-values:
1. Let this-key be a copy of key-stub. 1. If key-stub is an empty string, let this-key be a copy of
sorted-value.
2. Append sorted-value to this-key. 2. Otherwise:
1. Let this-key be a copy of key-stub.
2. Append a comma (",") to this-key.
3. Append sorted-value to this-key.
3. Let remaining-variants be a copy of all of the members of 3. Let remaining-variants be a copy of all of the members of
sorted-variants except the first. sorted-variants except the first.
4. If remaining-variants is empty, append this-key to possible- 4. If remaining-variants is empty, append this-key to possible-
keys. keys.
5. Else, run Find Available Keys on remaining-variants, this-key 5. Otherwise, run Find Available Keys on remaining-variants,
and possible-keys. this-key and possible-keys.
6. Return possible-keys. 3. Return possible-keys.
5.2. Example of Cache Behaviour 4.2. Check Vary
This algorithm is an example of how an implementation can meet the
requirement to apply the members of the Vary header field that are
not covered by Variants.
Given a stored response, stored-response:
1. Let filtered-vary be the field-value(s) of stored-response's
"Vary" header field.
2. Let processed-variants be a list containing the request header
fields that identify the content negotiation mechanisms supported
by the implementation.
3. Remove any member of filtered-vary that is a case-insensitive
match for a member of processed-variants.
4. If the secondary cache key (as calculated in [RFC7234],
Section 4.1) for stored_response matches incoming-request, using
filtered-vary for the value of the "Vary" response header, return
True.
5. Return False.
This returns a Boolean that indicates whether stored-response can be
used to satisfy the request.
Note that implementation of the Vary header field varies in practice,
and the algorithm above illustrates only one way to apply it. It is
equally viable to forward the request if there is a request header
listed in Vary but not Variants.
4.3. Example of Cache Behaviour
For example, if the selected variants-header was: For example, if the selected variants-header was:
Variants: Accept-Language;en;fr,de, Accept-Encoding;gzip,br Variants: Accept-Language;en;fr,de, Accept-Encoding;gzip,br
and the request contained the headers: and the request contained the headers:
Accept-Language: fr;q=1.0, en;q=0.1 Accept-Language: fr;q=1.0, en;q=0.1
Accept-Encoding: gzip Accept-Encoding: gzip
Then the sorted-variants would be: Then the sorted-variants would be:
[ [
["fr", "en"] // prefers French, will accept English ["fr", "en"] // prefers French, will accept English
["gzip", "identity"] // prefers gzip encoding, will accept identity ["gzip", "identity"] // prefers gzip encoding, will accept identity
] ]
Which means that the sorted-keys would be: Which means that the sorted-keys would be:
[ [
['fr', 'gzip'], 'fr gzip',
['fr', 'identity'], 'fr identity',
['en', 'gzip'], 'en gzip',
['en', 'identity'] 'en identity'
] ]
Representing a first preference of a French, gzip'd response. Thus, Representing a first preference of a French, gzip'd response. Thus,
if a cache has a response with: if a cache has a response with:
Variant-Key: fr, gzip Variant-Key: fr, gzip
it could be used to satisfy the first preference. If not, responses it could be used to satisfy the first preference. If not, responses
corresponding to the other keys could be returned, or the request corresponding to the other keys could be returned, or the request
could be forwarded towards the origin. could be forwarded towards the origin.
6. Example Headers 5. Origin Server Behaviour
6.1. Single Variant Origin servers that wish to take advantage of Variants will need to
generate both the Variants (Section 2) and Variant-Key (Section 3)
header fields in all cacheable responses for a given resource. If
either is omitted and the response is stored, it will have the effect
of disabling caching for that resource until it is no longer stored
(e.g., it expires, or is evicted).
Likewise, origin servers will need to assure that the members of both
header field values are in the same order and have the same length,
since discrepancies will cause caches to avoid using the responses
they occur in.
The value of the Variants header should be relatively stable for a
given resource over time; when it changes, it can have the effect of
invalidating previously stored responses.
As per Section 2.1, the Vary header is required to be set
appropriately when Variants is in use, so that caches that do not
implement this specification still operate correctly.
Origin servers are advised to carefully consider which content
negotiation mechanisms to enumerate in Variants; if a mechanism is
not supported by a receiving cache, it will "downgrade" to Vary
handling, which can negatively impact cache efficiency.
5.1. Examples
The operation of Variants is illustrated by the examples below.
5.1.1. Single Variant
Given a request/response pair: Given a request/response pair:
GET /foo HTTP/1.1 GET /clancy HTTP/1.1
Host: www.example.com Host: www.example.com
Accept-Language: en;q=1.0, fr;q=0.5 Accept-Language: en;q=1.0, fr;q=0.5
HTTP/1.1 200 OK HTTP/1.1 200 OK
Content-Type: image/gif Content-Type: image/gif
Content-Language: en Content-Language: en
Cache-Control: max-age=3600 Cache-Control: max-age=3600
Variants: Content-Language;en;de Variants: Content-Language;en;de
Variant-Key: en Variant-Key: en
Vary: Accept-Language Vary: Accept-Language
Transfer-Encoding: chunked Transfer-Encoding: chunked
Upon receipt of this response, the cache knows that two Upon receipt of this response, the cache knows that two
representations of this resource are available, one with a "Content- representations of this resource are available, one with a Content-
Language" of "en", and another whose "Content-Language" is "de". Language of "en", and another whose Content-Language is "de".
Subsequent requests (while this response is fresh) will cause the Subsequent requests (while this response is fresh) will cause the
cache to either reuse this response or forward the request, depending cache to either reuse this response or forward the request, depending
on what the selection algorithm determines. on what the selection algorithm determines.
So, if a request with "en" in "Accept-Language" is received and its So, if a request with "en" in Accept-Language is received and its
q-value indicates that it is acceptable, the stored response is used. q-value indicates that it is acceptable, the stored response is used.
A request that indicates that "de" is acceptable will be forwarded to A request that indicates that "de" is acceptable will be forwarded to
the origin, thereby populating the cache. A cache receiving a the origin, thereby populating the cache. A cache receiving a
request that indicates both languages are acceptable will use the request that indicates both languages are acceptable will use the
q-value to make a determination of what response to return. q-value to make a determination of what response to return.
A cache receiving a request that does not list either language as A cache receiving a request that does not list either language as
acceptable (or does not contain an Accept-Language at all) will acceptable (or does not contain an Accept-Language at all) will
return the "en" representation (possibly fetching it from the return the "en" representation (possibly fetching it from the
origin), since it is listed first in the "Variants" list. origin), since it is listed first in the Variants list.
Note that "Accept-Language" is listed in Vary, to assure backwards- Note that Accept-Language is listed in Vary, to assure backwards-
compatibility with caches that do not support "Variants". compatibility with caches that do not support Variants.
6.2. Multiple Variants 5.1.2. Multiple Variants
A more complicated request/response pair: A more complicated request/response pair:
GET /bar HTTP/1.1 GET /murray HTTP/1.1
Host: www.example.net Host: www.example.net
Accept-Language: en;q=1.0, fr;q=0.5 Accept-Language: en;q=1.0, fr;q=0.5
Accept-Encoding: gzip, br Accept-Encoding: gzip, br
HTTP/1.1 200 OK HTTP/1.1 200 OK
Content-Type: image/gif Content-Type: image/gif
Content-Language: en Content-Language: en
Content-Encoding: br Content-Encoding: br
Variants: Content-Language;en;jp;de Variants: Content-Language;en;jp;de
Variants: Content-Encoding;br;gzip Variants: Content-Encoding;br;gzip
Variant-Key: en, br Variant-Key: en, br
Vary: Accept-Language, Accept-Encoding Vary: Accept-Language, Accept-Encoding
Transfer-Encoding: chunked Transfer-Encoding: chunked
Here, the cache knows that there are two axes that the response Here, the cache knows that there are two axes that the response
varies upon; "Content-Language" and "Content-Encoding". Thus, there varies upon; Content-Language and Content-Encoding. Thus, there are
are a total of six possible representations for the resource, and the a total of nine possible representations for the resource (including
cache needs to consider the selection algorithms for both axes. the identity encoding), and the cache needs to consider the selection
algorithms for both axes.
Upon a subsequent request, if both selection algorithms return a Upon a subsequent request, if both selection algorithms return a
stored representation, it can be served from cache; otherwise, the stored representation, it can be served from cache; otherwise, the
request will need to be forwarded to origin. request will need to be forwarded to origin.
6.3. Partial Coverage 5.1.3. Partial Coverage
Now, consider the previous example, but where only one of the Vary'd Now, consider the previous example, but where only one of the Vary'd
axes is listed in "Variants": axes is listed in Variants:
GET /bar HTTP/1.1 GET /bar HTTP/1.1
Host: www.example.net Host: www.example.net
Accept-Language: en;q=1.0, fr;q=0.5 Accept-Language: en;q=1.0, fr;q=0.5
Accept-Encoding: gzip, br Accept-Encoding: gzip, br
HTTP/1.1 200 OK HTTP/1.1 200 OK
Content-Type: image/gif Content-Type: image/gif
Content-Language: en Content-Language: en
Content-Encoding: br Content-Encoding: br
skipping to change at page 12, line 18 skipping to change at page 14, line 4
Accept-Encoding: gzip, br Accept-Encoding: gzip, br
HTTP/1.1 200 OK HTTP/1.1 200 OK
Content-Type: image/gif Content-Type: image/gif
Content-Language: en Content-Language: en
Content-Encoding: br Content-Encoding: br
Variants: Content-Encoding;br;gzip Variants: Content-Encoding;br;gzip
Variant-Key: br Variant-Key: br
Vary: Accept-Language, Accept-Encoding Vary: Accept-Language, Accept-Encoding
Transfer-Encoding: chunked Transfer-Encoding: chunked
Here, the cache will need to calculate a secondary cache key as per Here, the cache will need to calculate a secondary cache key as per
[RFC7234], Section 4.1 - but considering only "Accept-Language" to be [RFC7234], Section 4.1 - but considering only Accept-Language to be
in its field-value - and then continue processing "Variants" for the in its field-value - and then continue processing Variants for the
set of stored responses that the algorithm described there selects. set of stored responses that the algorithm described there selects.
6. Defining Content Negotiation Using Variants
To be usable with Variants, proactive content negotiation mechanisms
need to be specified to take advantage of it. Specifically, they:
o MUST define a request header field that advertises the clients
preferences or capabilities, whose field-name SHOULD begin with
"Accept-".
o MUST define the syntax of available-values that will occur in
Variants and Variant-Key.
o MUST define an algorithm for selecting a result. It MUST return a
list of available-values that are suitable for the request, in
order of preference, given the value of the request header
nominated above and an available-values list from the Variants
header. If the result is an empty list, it implies that the cache
cannot satisfy the request.
Appendix A fulfils these requirements for some existing proactive
content negotiation mechanisms in HTTP.
7. IANA Considerations 7. IANA Considerations
This specification registers two values in the Permanent Message This specification registers two values in the Permanent Message
Header Field Names registry established by [RFC3864]: Header Field Names registry established by [RFC3864]:
o Header field name: Variants o Header field name: Variants
o Applicable protocol: http o Applicable protocol: http
o Status: standard o Status: standard
skipping to change at page 12, line 44 skipping to change at page 15, line 4
o Author/Change Controller: IETF o Author/Change Controller: IETF
o Specification document(s): [this document] o Specification document(s): [this document]
o Related information: o Related information:
o Header field name: Variant-Key o Header field name: Variant-Key
o Applicable protocol: http o Applicable protocol: http
o Status: standard o Status: standard
o Author/Change Controller: IETF o Author/Change Controller: IETF
o Specification document(s): [this document] o Specification document(s): [this document]
o Related information: o Related information:
8. Security Considerations 8. Security Considerations
If the number or advertised characteristics of the representations If the number or advertised characteristics of the representations
available for a resource are considered sensitive, the "Variants" available for a resource are considered sensitive, the Variants
header by its nature will leak them. header by its nature will leak them.
Note that the "Variants" header is not a commitment to make Note that the Variants header is not a commitment to make
representations of a certain nature available; the runtime behaviour representations of a certain nature available; the runtime behaviour
of the server always overrides hints like "Variants". of the server always overrides hints like Variants.
9. Acknowledgments 9. Acknowledgments
This protocol is conceptually similar to, but simpler than, This protocol is conceptually similar to, but simpler than,
Transparent Content Negotiation [RFC2295]. Thanks to its authors for Transparent Content Negotiation [RFC2295]. Thanks to its authors for
their inspiration. their inspiration.
It is also a generalisation of a Fastly VCL feature designed by It is also a generalisation of a Fastly VCL feature designed by
Rogier 'DocWilco' Mulhuijzen. Rogier 'DocWilco' Mulhuijzen.
skipping to change at page 14, line 18 skipping to change at page 16, line 28
<https://www.rfc-editor.org/info/rfc7234>. <https://www.rfc-editor.org/info/rfc7234>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>. May 2017, <https://www.rfc-editor.org/info/rfc8174>.
10.2. Informative References 10.2. Informative References
[I-D.ietf-httpbis-client-hints] [I-D.ietf-httpbis-client-hints]
Grigorik, I., "HTTP Client Hints", draft-ietf-httpbis- Grigorik, I., "HTTP Client Hints", draft-ietf-httpbis-
client-hints-04 (work in progress), April 2017. client-hints-05 (work in progress), January 2018.
[RFC2295] Holtman, K. and A. Mutz, "Transparent Content Negotiation [RFC2295] Holtman, K. and A. Mutz, "Transparent Content Negotiation
in HTTP", RFC 2295, DOI 10.17487/RFC2295, March 1998, in HTTP", RFC 2295, DOI 10.17487/RFC2295, March 1998,
<https://www.rfc-editor.org/info/rfc2295>. <https://www.rfc-editor.org/info/rfc2295>.
[RFC3864] Klyne, G., Nottingham, M., and J. Mogul, "Registration [RFC3864] Klyne, G., Nottingham, M., and J. Mogul, "Registration
Procedures for Message Header Fields", BCP 90, RFC 3864, Procedures for Message Header Fields", BCP 90, RFC 3864,
DOI 10.17487/RFC3864, September 2004, DOI 10.17487/RFC3864, September 2004,
<https://www.rfc-editor.org/info/rfc3864>. <https://www.rfc-editor.org/info/rfc3864>.
10.3. URIs 10.3. URIs
[1] https://github.com/mnot/I-D/labels/variants [1] https://github.com/mnot/I-D/labels/variants
[2] https://mnot.github.io/I-D/variants/ [2] https://mnot.github.io/I-D/variants/
[3] https://github.com/mnot/I-D/commits/gh-pages/variants [3] https://github.com/mnot/I-D/commits/gh-pages/variants
[4] https://datatracker.ietf.org/doc/draft-nottingham-variants/ [4] https://datatracker.ietf.org/doc/draft-nottingham-variants/
[5] https://github.com/mnot/variants-toy
Appendix A. Variants for Existing Content Negotiation Mechanisms Appendix A. Variants for Existing Content Negotiation Mechanisms
This appendix defines the required information to use existing This appendix defines the required information to use existing
proactive content negotiation mechanisms (as defined in [RFC7231], proactive content negotiation mechanisms (as defined in [RFC7231],
Section 5.3) with the "Variants" header field. Section 5.3) with the Variants header field.
A.1. Accept-Encoding A.1. Accept
This section defines handling for "Accept-Encoding" variants, as per This section defines handling for Accept variants, as per [RFC7231]
Section 5.3.2.
To perform content negotiation for Accept given a request-value and
available-values:
1. Let preferred-available be an empty list.
2. Let preferred-types be a list of the types in the request-value,
ordered by their weight, highest to lowest, as per [RFC7231]
Section 5.3.2 (omitting any coding with a weight of 0). If
"Accept" is not present or empty, preferred-types will be empty.
If a type lacks an explicit weight, an implementation MAY assign
one.
3. If the first member of available-values is not a member of
preferred-types, append it to preferred-types (thus making it the
default).
4. For each preferred-type in preferred-types:
1. If any member of available-values matches preferred-type,
using the media-range matching mechanism specified in
[RFC7231] Section 5.3.2 (which is case-insensitive), append
those members of available-values to preferred-available
(preserving the precedence order implied by the media ranges'
specificity).
5. Return preferred-available.
Note that this algorithm explicitly ignores extension parameters on
media types (e.g., "charset").
A.2. Accept-Encoding
This section defines handling for Accept-Encoding variants, as per
[RFC7231] Section 5.3.4. [RFC7231] Section 5.3.4.
To perform content negotiation for Accept-Encoding given an request- To perform content negotiation for Accept-Encoding given a request-
value and available-values: value and available-values:
1. Let preferred-codings be a list of the codings in the request- 1. Let preferred-available be an empty list.
2. Let preferred-codings be a list of the codings in the request-
value, ordered by their weight, highest to lowest, as per value, ordered by their weight, highest to lowest, as per
[RFC7231] Section 5.3.1 (omitting any coding with a weight of 0). [RFC7231] Section 5.3.1 (omitting any coding with a weight of 0).
If "Accept-Encoding" is not present or empty, preferred-codings If "Accept-Encoding" is not present or empty, preferred-codings
will be empty. will be empty. If a coding lacks an explicit weight, an
implementation MAY assign one.
2. If "identity" is not a member of preferred-codings, append 3. If "identity" is not a member of preferred-codings, append
"identity". "identity".
3. Append "identity" to available-values. 4. Append "identity" to available-values.
4. Remove any member of available-values not present in preferred- 5. For each preferred-coding in preferred-codings:
codings, comparing in a case-insensitive fashion.
5. Return available-values. 1. If there is a case-insensitive, character-for-character match
for preferred-coding in available-values, append that member
of available-values to preferred-available.
A.2. Accept-Language 6. Return preferred-available.
This section defines handling for "Accept-Language" variants, as per Note that the unencoded variant needs to have a Variant-Key header
field with a value of "identity".
A.3. Accept-Language
This section defines handling for Accept-Language variants, as per
[RFC7231] Section 5.3.5. [RFC7231] Section 5.3.5.
To perform content negotiation for Accept-Language given an request- To perform content negotiation for Accept-Language given a request-
value and available-values: value and available-values:
1. Let preferred-langs be a list of the language-ranges in the 1. Let preferred-available be an empty list.
2. Let preferred-langs be a list of the language-ranges in the
request-value, ordered by their weight, highest to lowest, as per request-value, ordered by their weight, highest to lowest, as per
[RFC7231] Section 5.3.1 (omitting any language-range with a [RFC7231] Section 5.3.1 (omitting any language-range with a
weight of 0). weight of 0). If a language-range lacks a weight, an
implementation MAY assign one.
2. If preferred-langs is empty, append "*". 3. If the first member of available-values is not a member of
preferred-langs, append it to preferred-langs (thus making it the
default).
3. Filter available-values using preferred-langs with either the 4. For each preferred-lang in preferred-langs:
Basic Filtering scheme defined in [RFC4647] Section 3.3.1, or the
Lookup scheme defined in Section 3.4 of that document. Use the
first member of available-values as the default.
4. Return available-values. 1. If any member of available-values matches preferred-lang,
using either the Basic or Extended Filtering scheme defined
in [RFC4647] Section 3.3, append those members of available-
values to preferred-available (preserving their order).
5. Return preferred-available.
Author's Address Author's Address
Mark Nottingham Mark Nottingham
Fastly Fastly
Email: mnot@mnot.net Email: mnot@mnot.net
URI: https://www.mnot.net/ URI: https://www.mnot.net/
 End of changes. 92 change blocks. 
191 lines changed or deleted 359 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/