The Key HTTP Response Header FieldAdobe Systems Incorporatedfielding@gbiv.comhttp://roy.gbiv.com/mnot@mnot.nethttp://www.mnot.net/
General
httpbis Working Groupcontent negotiationThe ‘Key’ header field for HTTP responses allows an origin server to describe the secondary cache
key (, section 4.1) for a resource, by conveying what is effectively a short algorithm
that can be used upon later requests to determine if a stored response is reusable for a given
request.Key has the advantage of avoiding an additional round trip for validation whenever a new request
differs slightly, but not significantly, from prior requests.Key also informs user agents of the request characteristics that might result in different content,
which can be useful if the user agent is not sending request header fields in order to reduce the
risk of fingerprinting.The issues list for this draft can be found at https://github.com/httpwg/http-extensions/labels/key.In HTTP caching , the Vary response header field effectively modifies the key used to
store and access a response to include information from the request’s headers. This “secondary
cache key” allows proactive content negotiation to work with caches.Vary’s operation is generic; it works well when caches understand the semantics of the selecting
headers. For example, the Accept-Language request header field has a well-defined syntax for
expressing the client’s preferences; a cache that understands this header field can select the
appropriate response (based upon its Content-Language header field) and serve it to a client,
without any knowledge of the underlying resource.Vary does not work as well when the criteria for selecting a response are specific to the resource.
For example, if the nature of the response depends upon the presence or absence of a particular
Cookie () in a request, Vary doesn’t have a mechanism to offer enough fine-grained,
resource-specific information to aid a cache’s selection of the appropriate response.This document defines a new response header field, “Key”, that allows resources to describe the
secondary cache key in a fine-grained, resource-specific manner, leading to improved cache
efficiency when responses depend upon such headers.For example, this response header field:indicates that the selected response depends upon the “_sess” and “ID” cookie values.This Key:indicates that there are two possible secondary cache keys for this resource; one for requests
whose User-Agent header field contains “MSIE”, and another for those that don’t.A more complex example:indicates that the selected response depends on the presence of two strings in the User-Agent
request header field, as well as the value of the “ID” cookie request header field.The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”,
“RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in
.This document uses the Augmented Backus-Naur Form (ABNF) notation of (including the
DQUOTE rule), and the list rule extension defined in , Section 7. It includes by
reference the field-name, quoted-string and quoted-pair rules from that document, and the parameter
rule from .The “Key” response header field describes the portions of the request that the resource currently
uses to select representations.As such, its semantics are similar to the “Vary” response header field, but it allows more
fine-grained description, using “key parameters”.Caches can use this information as part of determining whether a stored response can be used to
satisfy a given request. When a cache knows and fully understands the Key header field for a given
resource, it MAY ignore the Vary response header field in any stored responses for it.Additionally, user agents can use Key to discover if additional request header fields might
influence the resource’s selection of responses.The Key field-value is a comma-delimited list of selecting header fields (similar to Vary), with
zero to many parameters each, delimited by semicolons. Whitespace is not allowed in the field-value
between each field-name and its parameter set.Note that, as per , parameter names are case-insensitive, and parameter values can be
double-quoted strings (potentially with “"-escaped characters inside).The following header fields have the same effect:However, Key’s use of parameters allows:to indicate that the secondary cache key depends upon the Accept-Encoding header field and the
“foo” Cookie.One important difference between Vary and Key is how they are applied. Vary is specified to be
specific to the response it occurs within, whereas Key is specific to the resource (as identified
by the request URL) it is associated with. The most recent key you receive for a given resource is
applicable to all responses from that resource.This difference allows more efficient implementation (and reflects practices that many caches use
in implementing Vary already).This specification defines a selection of Key parameters to address common use cases such as
selection upon individual Cookie header fields, User-Agent substrings and numerical ranges. Future
parameters may define further capabilities.Origin servers SHOULD still send Vary when using Key, to ensure backwards compatibility.For example,Note that, in some cases, it may be better to explicitly use “Vary: *” if clients and caches don’t have any practical way to use the Vary header field’s value. For example,Except when Vary: * is used, the set of headers used in Key SHOULD reflect the same request header fields as Vary does, even if they don’t have parameters. For example,Here, Accept-Encoding is included in Key without parameters; caches MAY treat these as they do
values in the Vary header, relying upon knowledge of their generic semantics to select an
appropriate response.When used by a cache to determine whether a stored response can be used to satisfy a presented
request, each field-name in Key identifies a potential request header, just as with the Vary
response header field.However, each of these can have zero to many key parameters that change how the response selection
process (as defined in , Section 4.3)) works.In particular, when a cache fully implements this specification, it creates a secondary cache key
for every request by following the instructions in the Key header field, ignoring the Vary header
for this purpose.Then, when a new request is presented, the secondary cache key generated for that request can be
compared to the stored one to find the appropriate response, to determine if it can be selected.To generate a secondary cache key for a given request (including that which is stored with a
response) using Key, the following steps are taken:If the Key header field is not present on the most recent cacheable (as per , Section 3)) response seen for the resource, abort this algorithm (i.e., fall back to using Vary to determine the secondary cache key).Let key_value be the most recently seen Key header field value for the resource, as the result of Creating a Header Field Value ().Let secondary_key be an empty string.Create key_list by splitting key_value on “,” characters.For key_item in key_list: Remove any leading and trailing WSP from key_item.If key_item does not contain a “;” character, fail parameter processing () and skip to the next key_item.Let field_name be the string before the first “;” character in key_item.Let field_value be the result of Creating a Header Field Value () with field_name as the target_field_name and the request header list as header_list.Let parameters be the string after the first “;” character in key_item.Create param_list by splitting parameters on “;” characters, excepting “;” characters within quoted strings, as per Section 3.2.6.For parameter in param_list: If parameter does not contain a “=”, fail parameter processing () and skip to the next key_item.Let param_name be the string before the first “=” character in parameter, case-normalized to lowercase.If param_name does not identify a Key parameter processing algorithm that is implemented, fail parameter processing () and skip to the next key_item.Let param_value be the string after the first “=” character in parameter.If the first and last characters of param_value are both DQUOTE: Remove the first and last characters of param_value.Replace quoted-pairs within param_value with the octet following the backslash, as per Section 3.2.6.If param_value does not conform to the syntax defined for it by the parameter definition, fail parameter processing and skip to the next key_item.Run the identified processing algorithm on field_value with the param_value, and append the result to secondary_key. If parameter processing fails , skip to the next key_item.Append a separator character (e.g., NULL) to secondary_key.Return secondary_key.Note that this specification does not require that exact algorithm to be implemented. However,
implementations’ observable behavior MUST be identical to running it. This includes parameter
processing algorithms; implementations MAY use different internal artefacts for secondary cache
keys, as long as the results are the same.Likewise, while the secondary cache key associated with both stored and presented requests is
required to use the most recently seen Key header field for the resource in question, this can be
achieved using a variety of implementation strategies, including (but not limited to):Generating a new secondary cache key for every stored response associated with the resource upon each request.Caching the secondary cache key with the stored request/response pair and re-generating it when the Key header field is observed to change.Caching the secondary cache key with the stored response and invalidating the stored response(s) when the Key header field is observed to change.Given a header field name target_field_name and header_list, a list of (field_name,
field_value) tuples:Let target_field_values be an empty list.For each (field_name, field_value) tuple in header_list: If field_name does not match target_field_name, skip to the next tuple.Strip leading and trailing WSP from field_value and append it to target_field_values.If target_field_values is empty, return an empty string.Return the concatenation of target_field_values, separating each with “,” characters.In some cases, a key parameter cannot determine a secondary cache key corresponding to its
nominated header field value. When this happens, Key processing needs to fail safely, so that the
correct behavior is observed.When this happens, implementations MUST either behave as if the Key header was not present, or
assure that the nominated header fields being compared match, as per , Section 4.1.A Key parameter associates a name with a specific processing algorithm that takes two inputs; a
HTTP header value “header_value” (as described in ), and “parameter_value”, a string that
indicates how the identified header should be processed.The set of key parameters (and their associated processing algorithms) is extensible; see .
This document defines the following key parameters:The “div” parameter normalizes positive integer header values into groups by dividing them by a
configured value.Its value’s syntax is:To process a set of header fields against a div parameter, follow these steps (or their equivalent):If parameter_value is “0”, fail parameter processing .If header_value is the empty string, return “none”.If header_value contains a “,”, remove it and all subsequent characters.Remove all WSP characters from header_value.If header_value does not match the div ABNF rule, fail parameter processing ().Return the quotient of header_value / parameter_value (omitting the modulus).For example, the Key:indicates that the “Bar” header’s field value should be partitioned into groups of 5. Thus, the
following field values would be considered the same (because, divided by 5, they all result in 1):whereas these would be considered to be in a different group (because, divided by 5, they all result in 2);The “partition” parameter normalizes positive numeric header values into pre-defined segments.Its value’s syntax is:To process a set of header fields against a partition parameter, follow these steps (or their
equivalent):If header_value is the empty string, return “none”.If header_value contains a “,”, remove it and all subsequent characters.Remove all WSP characters from header_value.If header_value does not match the segment ABNF rule, fail parameter processing ().Let segment_id be 0.Create a list segment_list by splitting parameter_value on “:” characters.For each segment_value in segment_list: If header_value is less than segment_value when they are numerically compared, skip to step 7.Increment segment_id by 1.Return segment_id.For example, the Key:indicates that the “Foo” header’s field value should be divided into four segments:less than 2020 to less than 3030 to less than 40forty or greaterThus, the following headers would all be normalized to the first segment:whereas the following would fall into the second segment:The “match” parameter is used to determine if an exact value occurs in a list of header values. It
is case-sensitive.Its value’s syntax is:To process a set of header fields against a match parameter, follow these steps (or their
equivalent):If header_value is the empty string, return “none”.Create header_list by splitting header_value on “,” characters.For each header_item in header_list: Remove leading and trailing WSP characters in header_item.If the value of header_item is character-for-character identical to parameter_value, return “1”.Return “0”.For example, the Key:Would return “1” for the following header field values:and “0” for these:The “substr” parameter is used to determine if a value occurs as a substring of an item in a list
of header values. It is case-sensitive.Its value’s syntax is:To process a set of header fields against a substr parameter, follow these steps (or their
equivalent):If header_value is the empty string, return “none”.Create header_list by splitting header_value on “,” characters.For each header_item in header_list: Remove leading and trailing WSP characters in header_item.If the value of parameter_value is character-for-character present as a substring of header_value, return “1”.Return “0”.For example, the Key:Would return “1” for the following header field values:and “0” for these:The “param” parameter considers the request header field as a list of key=value parameters, and
uses the nominated key’s value as the secondary cache key.Its value’s syntax is:To process a list of header fields against a param parameter, follow these steps (or their
equivalent):Let header_list be an empty list.Create header_list_tmp1 by splitting header_value on “,” characters.For each header_item_tmp1 in header_list_tmp1: Create header_list_tmp2 by splitting header_item_tmp1 on “;” characters.For each header_item_tmp2 in header_list_tmp2: Remove leading and trailing WSP from header_item_tmp2.Append header_item_tmp2 to header_list.For each header_item in header_list: If the “=” character does not occur within header_item, skip to the next header_item.Let item_name be the string occurring before the first “=” character in header_item.If item_name does not case-insensitively match parameter_value, skip to the next header_item.Return the string occurring after the first “=” character in header_item.Return the empty string.Note that steps 2 and 3 accommodate semicolon-separated values, so that it can be used with the Cookie request header field.For example, the Key:The following headers would return the string (surrounded in single quotes) indicated:This specification defines the HTTP Key Parameter Registry, maintained at
http://www.iana.org/assignments/http-parameters/http-parameters.xhtml#key.Key Parameter registrations MUST include the following fields:Parameter Name: [name]Reference: [Pointer to specification text]Values to be added to this namespace require IETF Review (see Section 4.1 of [RFC5226]) and MUST
conform to the purpose of content coding defined in this section.This specification makes the following entries in the HTTP Key Parameter Registry:Parameter NameReferencedivpartitionmatchsubstrparamBecause Key is an alternative to Vary, it is possible for caches to behave differently based upon
whether they implement Key. Likewise, because support for any one Key parameter is not required, it
is possible for different implementations of Key to behave differently. In both cases, an attacker
might be able to exploit these differences.This risk is mitigated by the requirement to fall back to Vary when unsupported parameters are
encountered, coupled with the requirement that servers that use Key also include a relevant Vary
header.An attacker with the ability to inject response headers might be able to perform a cache poisoning
attack that tailors a response to a specific user (e.g., by Keying to a Cookie that’s specific to
them). While the attack is still possible without Key, the ability to tailor is new.When implemented, Key might result in a larger number of stored responses for a given resource in
caches; this, in turn, might be used to create an attack upon the cache itself. Good cache
replacement algorithms and denial of service monitoring in cache implementations are reasonable
mitigations against this risk.Key words for use in RFCs to Indicate Requirement LevelsIn many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements.Augmented BNF for Syntax Specifications: ABNFInternet technical specifications often need to define a formal syntax. Over the years, a modified version of Backus-Naur Form (BNF), called Augmented BNF (ABNF), has been popular among many Internet specifications. The current specification documents ABNF. It balances compactness and simplicity with reasonable representational power. The differences between standard BNF and ABNF involve naming rules, repetition, alternatives, order-independence, and value ranges. This specification also supplies additional rule definitions and encoding for a core lexical analyzer of the type common to several Internet specifications. [STANDARDS-TRACK]Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and RoutingThe Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems. This document provides an overview of HTTP architecture and its associated terminology, defines the "http" and "https" Uniform Resource Identifier (URI) schemes, defines the HTTP/1.1 message syntax and parsing requirements, and describes related security concerns for implementations.Hypertext Transfer Protocol (HTTP/1.1): Semantics and ContentThe Hypertext Transfer Protocol (HTTP) is a stateless \%application- level protocol for distributed, collaborative, hypertext information systems. This document defines the semantics of HTTP/1.1 messages, as expressed by request methods, request header fields, response status codes, and response header fields, along with the payload of messages (metadata and body content) and mechanisms for content negotiation.Hypertext Transfer Protocol (HTTP/1.1): CachingThe Hypertext Transfer Protocol (HTTP) is a stateless \%application- level protocol for distributed, collaborative, hypertext information systems. This document defines HTTP caches and the associated header fields that control cache behavior or indicate cacheable response messages.HTTP State Management MechanismThis document defines the HTTP Cookie and Set-Cookie header fields. These header fields can be used by HTTP servers to store state (called cookies) at HTTP user agents, letting the servers maintain a stateful session over the mostly stateless HTTP protocol. Although cookies have many historical infelicities that degrade their security and privacy, the Cookie and Set-Cookie header fields are widely used on the Internet. This document obsoletes RFC 2965. [STANDARDS-TRACK]Thanks to Ilya Grigorik, Amos Jeffries and Yoav Weiss for their feedback.