idnits 2.17.1 draft-ietf-httpbis-header-structure-15.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([2], [3], [4], [5], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 28, 2020) is 1550 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 1522 -- Looks like a reference, but probably isn't: '2' on line 1524 -- Looks like a reference, but probably isn't: '3' on line 1526 -- Looks like a reference, but probably isn't: '4' on line 1528 -- Looks like a reference, but probably isn't: '5' on line 1530 == Missing Reference: 'RFCxxxx' is mentioned on line 309, but not defined == Missing Reference: 'RFC3986' is mentioned on line 316, but not defined -- Looks like a reference, but probably isn't: '6' on line 1614 ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) -- Obsolete informational reference (is this intentional?): RFC 7231 (Obsoleted by RFC 9110) -- Obsolete informational reference (is this intentional?): RFC 7540 (Obsoleted by RFC 9113) Summary: 2 errors (**), 0 flaws (~~), 3 warnings (==), 9 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 HTTP M. Nottingham 3 Internet-Draft Fastly 4 Intended status: Standards Track P-H. Kamp 5 Expires: July 31, 2020 The Varnish Cache Project 6 January 28, 2020 8 Structured Headers for HTTP 9 draft-ietf-httpbis-header-structure-15 11 Abstract 13 This document describes a set of data types and associated algorithms 14 that are intended to make it easier and safer to define and handle 15 HTTP header fields. It is intended for use by specifications of new 16 HTTP header fields that wish to use a common syntax that is more 17 restrictive than traditional HTTP field values. 19 Note to Readers 21 _RFC EDITOR: please remove this section before publication_ 23 Discussion of this draft takes place on the HTTP working group 24 mailing list (ietf-http-wg@w3.org), which is archived at 25 https://lists.w3.org/Archives/Public/ietf-http-wg/ [1]. 27 Working Group information can be found at https://httpwg.github.io/ 28 [2]; source code and issues list for this draft can be found at 29 https://github.com/httpwg/http-extensions/labels/header-structure 30 [3]. 32 Tests for implementations are collected at https://github.com/httpwg/ 33 structured-header-tests [4]. 35 Implementations are tracked at https://github.com/httpwg/wiki/wiki/ 36 Structured-Headers [5]. 38 Status of This Memo 40 This Internet-Draft is submitted in full conformance with the 41 provisions of BCP 78 and BCP 79. 43 Internet-Drafts are working documents of the Internet Engineering 44 Task Force (IETF). Note that other groups may also distribute 45 working documents as Internet-Drafts. The list of current Internet- 46 Drafts is at https://datatracker.ietf.org/drafts/current/. 48 Internet-Drafts are draft documents valid for a maximum of six months 49 and may be updated, replaced, or obsoleted by other documents at any 50 time. It is inappropriate to use Internet-Drafts as reference 51 material or to cite them other than as "work in progress." 53 This Internet-Draft will expire on July 31, 2020. 55 Copyright Notice 57 Copyright (c) 2020 IETF Trust and the persons identified as the 58 document authors. All rights reserved. 60 This document is subject to BCP 78 and the IETF Trust's Legal 61 Provisions Relating to IETF Documents 62 (https://trustee.ietf.org/license-info) in effect on the date of 63 publication of this document. Please review these documents 64 carefully, as they describe your rights and restrictions with respect 65 to this document. Code Components extracted from this document must 66 include Simplified BSD License text as described in Section 4.e of 67 the Trust Legal Provisions and are provided without warranty as 68 described in the Simplified BSD License. 70 Table of Contents 72 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 73 1.1. Intentionally Strict Processing . . . . . . . . . . . . . 4 74 1.2. Notational Conventions . . . . . . . . . . . . . . . . . 5 75 2. Defining New Structured Headers . . . . . . . . . . . . . . . 5 76 3. Structured Data Types . . . . . . . . . . . . . . . . . . . . 7 77 3.1. Lists . . . . . . . . . . . . . . . . . . . . . . . . . . 8 78 3.1.1. Inner Lists . . . . . . . . . . . . . . . . . . . . . 8 79 3.1.2. Parameters . . . . . . . . . . . . . . . . . . . . . 9 80 3.2. Dictionaries . . . . . . . . . . . . . . . . . . . . . . 10 81 3.3. Items . . . . . . . . . . . . . . . . . . . . . . . . . . 11 82 3.3.1. Integers . . . . . . . . . . . . . . . . . . . . . . 12 83 3.3.2. Decimals . . . . . . . . . . . . . . . . . . . . . . 12 84 3.3.3. Strings . . . . . . . . . . . . . . . . . . . . . . . 12 85 3.3.4. Tokens . . . . . . . . . . . . . . . . . . . . . . . 13 86 3.3.5. Byte Sequences . . . . . . . . . . . . . . . . . . . 13 87 3.3.6. Booleans . . . . . . . . . . . . . . . . . . . . . . 14 88 4. Working With Structured Headers in HTTP Headers . . . . . . . 14 89 4.1. Serializing Structured Headers . . . . . . . . . . . . . 14 90 4.1.1. Serializing a List . . . . . . . . . . . . . . . . . 15 91 4.1.2. Serializing a Dictionary . . . . . . . . . . . . . . 17 92 4.1.3. Serializing an Item . . . . . . . . . . . . . . . . . 17 93 4.1.4. Serializing an Integer . . . . . . . . . . . . . . . 18 94 4.1.5. Serializing a Decimal . . . . . . . . . . . . . . . . 19 95 4.1.6. Serializing a String . . . . . . . . . . . . . . . . 19 96 4.1.7. Serializing a Token . . . . . . . . . . . . . . . . . 20 97 4.1.8. Serializing a Byte Sequence . . . . . . . . . . . . . 20 98 4.1.9. Serializing a Boolean . . . . . . . . . . . . . . . . 21 99 4.2. Parsing Header Fields into Structured Headers . . . . . . 21 100 4.2.1. Parsing a List . . . . . . . . . . . . . . . . . . . 22 101 4.2.2. Parsing a Dictionary . . . . . . . . . . . . . . . . 24 102 4.2.3. Parsing an Item . . . . . . . . . . . . . . . . . . . 25 103 4.2.4. Parsing a Number . . . . . . . . . . . . . . . . . . 27 104 4.2.5. Parsing a String . . . . . . . . . . . . . . . . . . 28 105 4.2.6. Parsing a Token . . . . . . . . . . . . . . . . . . . 29 106 4.2.7. Parsing a Byte Sequence . . . . . . . . . . . . . . . 29 107 4.2.8. Parsing a Boolean . . . . . . . . . . . . . . . . . . 30 108 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 31 109 6. Security Considerations . . . . . . . . . . . . . . . . . . . 31 110 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 31 111 7.1. Normative References . . . . . . . . . . . . . . . . . . 31 112 7.2. Informative References . . . . . . . . . . . . . . . . . 32 113 7.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 32 114 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 33 115 Appendix B. Frequently Asked Questions . . . . . . . . . . . . . 33 116 B.1. Why not JSON? . . . . . . . . . . . . . . . . . . . . . . 33 117 B.2. Structured Headers don't "fit" my data. . . . . . . . . . 34 118 Appendix C. Implementation Notes . . . . . . . . . . . . . . . . 34 119 Appendix D. Changes . . . . . . . . . . . . . . . . . . . . . . 35 120 D.1. Since draft-ietf-httpbis-header-structure-14 . . . . . . 35 121 D.2. Since draft-ietf-httpbis-header-structure-13 . . . . . . 35 122 D.3. Since draft-ietf-httpbis-header-structure-12 . . . . . . 36 123 D.4. Since draft-ietf-httpbis-header-structure-11 . . . . . . 36 124 D.5. Since draft-ietf-httpbis-header-structure-10 . . . . . . 36 125 D.6. Since draft-ietf-httpbis-header-structure-09 . . . . . . 36 126 D.7. Since draft-ietf-httpbis-header-structure-08 . . . . . . 37 127 D.8. Since draft-ietf-httpbis-header-structure-07 . . . . . . 37 128 D.9. Since draft-ietf-httpbis-header-structure-06 . . . . . . 38 129 D.10. Since draft-ietf-httpbis-header-structure-05 . . . . . . 38 130 D.11. Since draft-ietf-httpbis-header-structure-04 . . . . . . 38 131 D.12. Since draft-ietf-httpbis-header-structure-03 . . . . . . 38 132 D.13. Since draft-ietf-httpbis-header-structure-02 . . . . . . 38 133 D.14. Since draft-ietf-httpbis-header-structure-01 . . . . . . 39 134 D.15. Since draft-ietf-httpbis-header-structure-00 . . . . . . 39 135 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 39 137 1. Introduction 139 Specifying the syntax of new HTTP header fields is an onerous task; 140 even with the guidance in Section 8.3.1 of [RFC7231], there are many 141 decisions - and pitfalls - for a prospective HTTP header field 142 author. 144 Once a header field is defined, bespoke parsers and serializers often 145 need to be written, because each header has slightly different 146 handling of what looks like common syntax. 148 This document introduces a set of common data structures for use in 149 definitions of new HTTP header field values to address these 150 problems. In particular, it defines a generic, abstract model for 151 header field values, along with a concrete serialisation for 152 expressing that model in HTTP [RFC7230] header fields. 154 HTTP headers that are defined as "Structured Headers" use the types 155 defined in this specification to define their syntax and basic 156 handling rules, thereby simplifying both their definition by 157 specification writers and handling by implementations. 159 Additionally, future versions of HTTP can define alternative 160 serialisations of the abstract model of these structures, allowing 161 headers that use it to be transmitted more efficiently without being 162 redefined. 164 Note that it is not a goal of this document to redefine the syntax of 165 existing HTTP headers; the mechanisms described herein are only 166 intended to be used with headers that explicitly opt into them. 168 Section 2 describes how to specify a Structured Header. 170 Section 3 defines a number of abstract data types that can be used in 171 Structured Headers. Those abstract types can be serialized into and 172 parsed from HTTP headers using the algorithms described in Section 4. 174 1.1. Intentionally Strict Processing 176 This specification intentionally defines strict parsing and 177 serialisation behaviours using step-by-step algorithms; the only 178 error handling defined is to fail the operation altogether. 180 It is designed to encourage faithful implementation and therefore 181 good interoperability. Therefore, an implementation that tried to be 182 "helpful" by being more tolerant of input would make interoperability 183 worse, since that would create pressure on other implementations to 184 implement similar (but likely subtly different) workarounds. 186 In other words, strict processing is an intentional feature of this 187 specification; it allows non-conformant input to be discovered and 188 corrected by the producer early, and avoids both interoperability and 189 security issues that might otherwise result. 191 Note that as a result of this strictness, if a header field is 192 appended to by multiple parties (e.g., intermediaries, or different 193 components in the sender), an error in one party's value is likely to 194 cause the entire header field to fail parsing. 196 1.2. Notational Conventions 198 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 199 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 200 "OPTIONAL" in this document are to be interpreted as described in BCP 201 14 [RFC2119] [RFC8174] when, and only when, they appear in all 202 capitals, as shown here. 204 This document uses algorithms to specify parsing and serialisation 205 behaviours, and the Augmented Backus-Naur Form (ABNF) notation of 206 [RFC5234] to illustrate expected syntax in HTTP header fields. In 207 doing so, uses the VCHAR, SP, DIGIT, ALPHA and DQUOTE rules from 208 [RFC5234]. It also includes the tchar rule from [RFC7230]. 210 When parsing from HTTP header fields, implementations MUST follow the 211 algorithms, but MAY vary in implementation so as the behaviours are 212 indistinguishable from specified behaviour. If there is disagreement 213 between the parsing algorithms and ABNF, the specified algorithms 214 take precedence. In some places, the algorithms are "greedy" with 215 whitespace, but this should not affect conformance. 217 For serialisation to header fields, the ABNF illustrates the range of 218 acceptable wire representations with as much fidelity as possible, 219 and the algorithms define the recommended way to produce them. 220 Implementations MAY vary from the specified behaviour so long as the 221 output still matches the ABNF. 223 2. Defining New Structured Headers 225 To specify a HTTP header as a structured header, its authors needs 226 to: 228 o Reference this specification. Recipients and generators of the 229 header need to know that the requirements of this document are in 230 effect. 232 o Specify the type of the header field itself; either Dictionary 233 (Section 3.2), List (Section 3.1), or Item (Section 3.3). 235 o Define the semantics of those structures. 237 o Specify any additional constraints upon the structures used, as 238 well as the consequences when those constraints are violated. 240 Typically, this means that a header definition will specify the top- 241 level type - Dictionary, List or Item - and then define its allowable 242 types, and constraints upon them. For example, a header defined as a 243 List might have all Integer members, or a mix of types; a header 244 defined as an Item might allow only Strings, and additionally only 245 strings beginning with the letter "Q". Likewise, inner lists are 246 only valid when a header definition explicitly allows them. 248 When Structured Headers parsing fails, the header is ignored (see 249 Section 4.2); in most situations, violating header-specific 250 constraints should have the same effect. Thus, if a header is 251 defined as an Item and required to be an Integer, but a String is 252 received, it will by default be ignored. If the header requires 253 different error handling, this should be explicitly specified. 255 However, both items and inner lists allow parameters as an 256 extensibility mechanism; this means that values can later be extended 257 to accommodate more information, if need be. As a result, header 258 specifications are discouraged from defining the presence of an 259 unrecognised parameter as an error condition. 261 To help assure that this extensibility is available in the future, 262 and to encourage consumers to use a fully capable Structured Headers 263 parser, a header definition can specify that "grease" parameters be 264 added by senders. For example, a specification could stipulate that 265 all parameters beginning with the letter 'q' are reserved for this 266 use. 268 Note that a header field definition cannot relax the requirements of 269 this specification because doing so would preclude handling by 270 generic software; they can only add additional constraints (for 271 example, on the numeric range of integers and decimals, the format of 272 strings and tokens, the types allowed in a dictionary's values, or 273 the number of items in a list). Likewise, header field definitions 274 can only use Structured Headers for the entire header field value, 275 not a portion thereof. 277 This specification defines minimums for the length or number of 278 various structures supported by Structured Headers implementations. 279 It does not specify maximum sizes in most cases, but header authors 280 should be aware that HTTP implementations do impose various limits on 281 the size of individual header fields, the total number of fields, 282 and/or the size of the entire header block. 284 Specifications can refer to a Structured Header's field-name as a 285 "structured header name" and its field-value as a "structured header 286 value" as necessary. Header definitions are encouraged to use the 287 ABNF rules beginning with "sh-" defined in this specification; other 288 rules in this specification are not intended for their use. 290 For example, a fictitious Foo-Example header field might be specified 291 as: 293 42. Foo-Example Header 295 The Foo-Example HTTP header field conveys information about how 296 much Foo the message has. 298 Foo-Example is a Item Structured Header [RFCxxxx]. Its value MUST be 299 an Integer (Section Y.Y of [RFCxxxx]). Its ABNF is: 301 Foo-Example = sh-integer 303 Its value indicates the amount of Foo in the message, and MUST 304 be between 0 and 10, inclusive; other values MUST cause 305 the entire header to be ignored. 307 The following parameters are defined: 308 * A parameter whose name is "fooUrl", and whose value is a string 309 (Section Y.Y of [RFCxxxx]), conveying the Foo URLs 310 for the message. See below for processing requirements. 312 "fooUrl" contains a URI-reference (Section 4.1 of 313 [RFC3986], Section 4.1). If its value is not a valid URI-reference, 314 that URL MUST be ignored. If its value is a relative reference 315 (Section 4.2 of [RFC3986]), it MUST be resolved (Section 5 of 316 [RFC3986]) before being used. 318 For example: 320 Foo-Example: 2; foourl="https://foo.example.com/" 322 3. Structured Data Types 324 This section defines the abstract value types that can be composed 325 into Structured Headers. The ABNF provided represents the on-wire 326 format in HTTP headers. 328 In summary: 330 o There are three top-level types that a HTTP header can be defined 331 as; Lists, Dictionaries, and Items. 333 o Lists and Dictionaries are containers; their members can be Items 334 or Inner Lists (which are themselves lists of items). 336 o Both Items and Inner Lists can be parameterised with key/value 337 pairs. 339 3.1. Lists 341 Lists are arrays of zero or more members, each of which can be an 342 item (Section 3.3) or an inner list (Section 3.1.1), both of which 343 can be parameterised (Section 3.1.2). 345 The ABNF for lists in HTTP headers is: 347 sh-list = list-member *( *SP "," *SP list-member ) 348 list-member = sh-item / inner-list 350 In HTTP headers, each member is separated by a comma and optional 351 whitespace. For example, a header field whose value is defined as a 352 list of strings could look like: 354 Example-StrListHeader: "foo", "bar", "It was the best of times." 356 In HTTP headers, an empty list is denoted by not serialising the 357 header at all. 359 Note that lists can have their members split across multiple 360 instances inside a block of fields; for example, the following are 361 equivalent: 363 Example-Hdr: foo, bar 365 and 367 Example-Hdr: foo 368 Example-Hdr: bar 370 However, members of a list cannot be safely split between instances; 371 see Section 4.2 for details. 373 Parsers MUST support lists containing at least 1024 members. Header 374 specifications can constrain the types and cardinality of individual 375 list values as they require. 377 3.1.1. Inner Lists 379 An inner list is an array of zero or more items (Section 3.3). Both 380 the individual items and the inner-list itself can be parameterised 381 (Section 3.1.2). 383 The ABNF for inner-lists in HTTP headers is: 385 inner-list = "(" *SP [ sh-item *( 1*SP sh-item ) *SP ] ")" 386 *parameter 388 In HTTP headers, inner lists are denoted by surrounding parenthesis, 389 and have their values delimited by a single space. A header field 390 whose value is defined as a list of inner-lists of strings could look 391 like: 393 Example-StrListListHeader: ("foo" "bar"), ("baz"), ("bat" "one"), () 395 Note that the last member in this example is an empty inner list. 397 A header field whose value is defined as a list of inner-lists with 398 parameters at both levels could look like: 400 Example-ListListParam: ("foo"; a=1;b=2);lvl=5, ("bar" "baz");lvl=1 402 Parsers MUST support inner-lists containing at least 256 members. 403 Header specifications can constrain the types and cardinality of 404 individual inner-list members as they require. 406 3.1.2. Parameters 408 Parameters are an ordered map of key-values pairs that are associated 409 with an item (Section 3.3) or inner-list (Section 3.1.1). The keys 410 are unique within the scope of a map of parameters, and the values 411 are bare items (i.e., they themselves cannot be parameterised; see 412 Section 3.3). 414 The ABNF for parameters in HTTP headers is: 416 parameter = ";" *SP param-name [ "=" param-value ] 417 param-name = key 418 key = lcalpha *( lcalpha / DIGIT / "_" / "-" / "." / "*" ) 419 lcalpha = %x61-7A ; a-z 420 param-value = bare-item 422 In HTTP headers, parameters are separated from their item or inner- 423 list and each other by semicolons. For example: 425 Example-ParamListHeader: abc;a=1;b=2; cde_456, (ghi;jk=4 l);q="9";r=w 427 Parameters whose value is Boolean true MUST omit that value when 428 serialised. For example: 430 Example-IntHeader: 1; a; b=?0 431 Note that this requirement is only on serialisation; parsers are 432 still required to correctly handle the true value when it appears in 433 parameters. 435 Parsers MUST support at least 256 parameters on an item or inner- 436 list, and support parameter keys with at least 64 characters. Header 437 specifications can constrain the types and cardinality of individual 438 parameter names and values as they require. 440 3.2. Dictionaries 442 Dictionaries are ordered maps of name-value pairs, where the names 443 are short, textual strings and the values are items (Section 3.3) or 444 arrays of items, both of which can be parameterised (Section 3.1.2). 445 There can be zero or more members, and their names are unique in the 446 scope of the dictionary they occur within. 448 Implementations MUST provide access to dictionaries both by index and 449 by name. Specifications MAY use either means of accessing the 450 members. 452 The ABNF for dictionaries in HTTP headers is: 454 sh-dictionary = dict-member *( *SP "," *SP dict-member ) 455 dict-member = member-name [ "=" member-value ] 456 member-name = key 457 member-value = sh-item / inner-list 459 In HTTP headers, members are separated by a comma with optional 460 whitespace, while names and values are separated by "=" (without 461 whitespace). For example: 463 Example-DictHeader: en="Applepie", da=:w4ZibGV0w6ZydGU=: 465 Members whose value is Boolean true MUST omit that value when 466 serialised, unless it has parameters. For example, here both "b" and 467 "c" are true, but "c"'s value is serialised because it has 468 parameters: 470 Example-DictHeader: a=?0, b, c=?1; foo=bar 472 Note that this requirement is only on serialisation; parsers are 473 still required to correctly handle the true value when it appears in 474 dictionary values. 476 A dictionary with a member whose value is an inner-list of tokens: 478 Example-DictListHeader: rating=1.5, feelings=(joy sadness) 479 A dictionary with a mix of singular and list values, some with 480 parameters: 482 Example-MixDict: a=(1 2), b=3, c=4;aa=bb, d=(5 6);valid=?1 484 As with lists, an empty dictionary is represented in HTTP headers by 485 omitting the entire header field. 487 Typically, a header field specification will define the semantics of 488 dictionaries by specifying the allowed type(s) for individual member 489 names, as well as whether their presence is required or optional. 490 Recipients MUST ignore names that are undefined or unknown, unless 491 the header field's specification specifically disallows them. 493 Note that dictionaries can have their members split across multiple 494 instances inside a block of fields; for example, the following are 495 equivalent: 497 Example-Hdr: foo=1, bar=2 499 and 501 Example-Hdr: foo=1 502 Example-Hdr: bar=2 504 However, members of a dictionary cannot be safely split between 505 instances; see Section 4.2 for details. 507 Parsers MUST support dictionaries containing at least 1024 name/value 508 pairs, and names with at least 64 characters. 510 3.3. Items 512 An item is can be a integer (Section 3.3.1), decimal (Section 3.3.2), 513 string (Section 3.3.3), token (Section 3.3.4), byte sequence 514 (Section 3.3.5), or Boolean (Section 3.3.6). It can have associated 515 parameters (Section 3.1.2). 517 The ABNF for items in HTTP headers is: 519 sh-item = bare-item *parameter 520 bare-item = sh-integer / sh-decimal / sh-string / sh-token / sh-binary 521 / sh-boolean 523 For example, a header field that is defined to be an Item that is an 524 integer might look like: 526 Example-IntItemHeader: 5 527 or with parameters: 529 Example-IntItemHeader: 5; foo=bar 531 3.3.1. Integers 533 Integers have a range of -999,999,999,999,999 to 999,999,999,999,999 534 inclusive (i.e., up to fifteen digits, signed), for IEEE 754 535 compatibility ([IEEE754]). 537 The ABNF for integers in HTTP headers is: 539 sh-integer = ["-"] 1*15DIGIT 541 For example: 543 Example-IntegerHeader: 42 545 Note that commas in integers are used in this section's prose only 546 for readability; they are not valid in the wire format. 548 3.3.2. Decimals 550 Decimals are numbers with an integer and a fractional component. The 551 Integer component has at most 12 digits; the fractional component has 552 at most three digits. 554 The ABNF for decimals in HTTP headers is: 556 sh-decimal = ["-"] 1*12DIGIT "." 1*3DIGIT 558 For example, a header whose value is defined as a decimal could look 559 like: 561 Example-DecimalHeader: 4.5 563 3.3.3. Strings 565 Strings are zero or more printable ASCII [RFC0020] characters (i.e., 566 the range %x20 to %x7E). Note that this excludes tabs, newlines, 567 carriage returns, etc. 569 The ABNF for strings in HTTP headers is: 571 sh-string = DQUOTE *(chr) DQUOTE 572 chr = unescaped / escaped 573 unescaped = %x20-21 / %x23-5B / %x5D-7E 574 escaped = "\" ( DQUOTE / "\" ) 575 In HTTP headers, strings are delimited with double quotes, using a 576 backslash ("\") to escape double quotes and backslashes. For 577 example: 579 Example-StringHeader: "hello world" 581 Note that strings only use DQUOTE as a delimiter; single quotes do 582 not delimit strings. Furthermore, only DQUOTE and "\" can be 583 escaped; other characters after "\" MUST cause parsing to fail. 585 Unicode is not directly supported in strings, because it causes a 586 number of interoperability issues, and - with few exceptions - header 587 values do not require it. 589 When it is necessary for a field value to convey non-ASCII content, a 590 byte sequence (Section 3.3.5) SHOULD be specified, along with a 591 character encoding (preferably [UTF-8]). 593 Parsers MUST support strings with at least 1024 characters. 595 3.3.4. Tokens 597 Tokens are short textual words; their abstract model is identical to 598 their expression in the HTTP header serialisation. 600 The ABNF for tokens in HTTP headers is: 602 sh-token = ( ALPHA / "\*" ) *( tchar / ":" / "/" ) 604 Parsers MUST support tokens with at least 512 characters. 606 Note that a Structured Header token allows the characters as the 607 "token" ABNF rule defined in [RFC7230], with the exceptions that the 608 first character is required to be either ALPHA or "*", and ":" and 609 "/" are also allowed in subsequent characters. 611 3.3.5. Byte Sequences 613 Byte sequences can be conveyed in Structured Headers. 615 The ABNF for a byte sequence in HTTP headers is: 617 sh-binary = ":" *(base64) ":" 618 base64 = ALPHA / DIGIT / "+" / "/" / "=" 620 In HTTP headers, a byte sequence is delimited with colons and encoded 621 using base64 ([RFC4648], Section 4). For example: 623 Example-BinaryHdr: :cHJldGVuZCB0aGlzIGlzIGJpbmFyeSBjb250ZW50Lg==: 625 Parsers MUST support byte sequences with at least 16384 octets after 626 decoding. 628 3.3.6. Booleans 630 Boolean values can be conveyed in Structured Headers. 632 The ABNF for a Boolean in HTTP headers is: 634 sh-boolean = "?" boolean 635 boolean = "0" / "1" 637 In HTTP headers, a boolean is indicated with a leading "?" character 638 followed by a "1" for a true value or "0" for false. For example: 640 Example-BoolHdr: ?1 642 4. Working With Structured Headers in HTTP Headers 644 This section defines how to serialize and parse Structured Headers in 645 header fields, and protocols compatible with them (e.g., in HTTP/2 646 [RFC7540] before HPACK [RFC7541] is applied). 648 4.1. Serializing Structured Headers 650 Given a structure defined in this specification, return an ASCII 651 string suitable for use in a HTTP header value. 653 1. If the structure is a Dictionary or List and its value is empty 654 (i.e., it has no members), do not serialize the field at all 655 (i.e., omit both the field-name and field-value). 657 2. If the structure is a Dictionary, let output_string be the result 658 of running Serializing a Dictionary (Section 4.1.2) with the 659 structure. 661 3. Else if the structure is a List, let output_string be the result 662 of running Serializing a List (Section 4.1.1) with the structure. 664 4. Else if the structure is an Item, let output_string be the result 665 of running Serializing an Item (Section 4.1.3) with the 666 structure. 668 5. Else, fail serialisation. 670 6. Return output_string converted into an array of bytes, using 671 ASCII encoding [RFC0020]. 673 4.1.1. Serializing a List 675 Given an array of (member_value, parameters) tuples as input_list, 676 return an ASCII string suitable for use in a HTTP header value. 678 1. Let output be an empty string. 680 2. For each (member_value, parameters) of input_list: 682 1. If member_value is an array, append the result of running 683 Serialising an Inner List (Section 4.1.1.1) with 684 (member_value, parameters) to output. 686 2. Otherwise, append the result of running Serializing an Item 687 (Section 4.1.3) with (member_value, parameters) to output. 689 3. If more member_values remain in input_list: 691 1. Append a COMMA to output. 693 2. Append a single SP to output. 695 3. Return output. 697 4.1.1.1. Serialising an Inner List 699 Given an array of (member_value, parameters) tuples as inner_list, 700 and parameters as list_parameters, return an ASCII string suitable 701 for use in a HTTP header value. 703 1. Let output be the string "(". 705 2. For each (member_value, parameters) of inner_list: 707 1. Append the result of running Serializing an Item 708 (Section 4.1.3) with (member_value, parameters) to output. 710 2. If more values remain in inner_list, append a single SP to 711 output. 713 3. Append ")" to output. 715 4. Append the result of running Serializing Parameters 716 Section 4.1.1.2 with list_parameters to output. 718 5. Return output. 720 4.1.1.2. Serializing Parameters 722 Given an ordered dictionary as input_parameters (each member having a 723 param_name and a param_value), return an ASCII string suitable for 724 use in a HTTP header value. 726 1. Let output be an empty string. 728 2. For each parameter-name with a value of param_value in 729 input_parameters: 731 1. Append ";" to output. 733 2. Append the result of running Serializing a Key 734 (Section 4.1.1.3) with param_name to output. 736 3. If param_value is not Boolean true: 738 1. Append "=" to output. 740 2. Append the result of running Serializing a bare Item 741 (Section 4.1.3.1) with param_value to output. 743 3. Return output. 745 4.1.1.3. Serializing a Key 747 Given a key as input_key, return an ASCII string suitable for use in 748 a HTTP header value. 750 1. If input_key is not a sequence of characters, or contains 751 characters not in lcalpha, DIGIT, "_", "-", ".", or "*" fail 752 serialisation. 754 2. If the first character of input_key is not lcalpha, fail parsing. 756 3. Let output be an empty string. 758 4. Append input_key to output. 760 5. Return output. 762 4.1.2. Serializing a Dictionary 764 Given an ordered dictionary as input_dictionary (each member having a 765 member_name and a tuple value of (member_value, parameters)), return 766 an ASCII string suitable for use in a HTTP header value. 768 1. Let output be an empty string. 770 2. For each member_name with a value of (member_value, parameters) 771 in input_dictionary: 773 1. Append the result of running Serializing a Key 774 (Section 4.1.1.3) with member's member_name to output. 776 3. If member_value is not Boolean true or parameters is not empty: 778 1. Append "=" to output. 780 1. If member_value is an array, append the result of running 781 Serialising an Inner List (Section 4.1.1.1) with 782 (member_value, parameters) to output. 784 2. Otherwise, append the result of running Serializing an 785 Item (Section 4.1.3) with (member_value, parameters) to 786 output. 788 4. If more members remain in input_dictionary: 790 1. Append a COMMA to output. 792 2. Append a single SP to output. 794 5. Return output. 796 4.1.3. Serializing an Item 798 Given an item bare_item and parameters item_parameters as input, 799 return an ASCII string suitable for use in a HTTP header value. 801 1. Let output be an empty string. 803 2. Append the result of running Serializing a Bare Item 804 Section 4.1.3.1 with bare_item to output. 806 3. Append the result of running Serializing Parameters 807 Section 4.1.1.2 with item_parameters to output. 809 4. Return output. 811 4.1.3.1. Serialising a Bare Item 813 Given an item as input_item, return an ASCII string suitable for use 814 in a HTTP header value. 816 1. If input_item is an integer, return the result of running 817 Serializing an Integer (Section 4.1.4) with input_item. 819 2. If input_item is a decimal, return the result of running 820 Serializing a Decimal (Section 4.1.5) with input_item. 822 3. If input_item is a string, return the result of running 823 Serializing a String (Section 4.1.6) with input_item. 825 4. If input_item is a token, return the result of running 826 Serializing a Token (Section 4.1.7) with input_item. 828 5. If input_item is a Boolean, return the result of running 829 Serializing a Boolean (Section 4.1.9) with input_item. 831 6. If input_item is a byte sequence, return the result of running 832 Serializing a Byte Sequence (Section 4.1.8) with input_item. 834 7. Otherwise, fail serialisation. 836 4.1.4. Serializing an Integer 838 Given an integer as input_integer, return an ASCII string suitable 839 for use in a HTTP header value. 841 1. If input_integer is not an integer in the range of 842 -999,999,999,999,999 to 999,999,999,999,999 inclusive, fail 843 serialisation. 845 2. Let output be an empty string. 847 3. If input_integer is less than (but not equal to) 0, append "-" to 848 output. 850 4. Append input_integer's numeric value represented in base 10 using 851 only decimal digits to output. 853 5. Return output. 855 4.1.5. Serializing a Decimal 857 Given a decimal_number as input_decimal, return an ASCII string 858 suitable for use in a HTTP header value. 860 1. Let output be an empty string. 862 2. If input_decimal is less than (but not equal to) 0, append "-" to 863 output. 865 3. Append input_decimal's integer component represented in base 10 866 (using only decimal digits) to output; if it is zero, append "0". 868 4. If the number of characters appended in the previous step is 869 greater than 12, fail serialisation. 871 5. Append "." to output. 873 6. If input_decimal's fractional component is zero, append "0" to 874 output. 876 7. Else if input_decimal's fractional component has up to three 877 digits, append them represented in base 10 (using only decimal 878 digits) to output. 880 8. Otherwise, append the first three digits of input_decimal's 881 fractional component (represented in base 10, using only decimal 882 digits) to output, rounding the final digit to the nearest value, 883 or to the even value if it is equidistant. 885 9. Return output. 887 4.1.6. Serializing a String 889 Given a string as input_string, return an ASCII string suitable for 890 use in a HTTP header value. 892 1. If input_string is not a sequence of characters, or contains 893 characters in the range %x00-1f or %x7f (i.e., is not in VCHAR or 894 SP), fail serialisation. 896 2. Let output be an empty string. 898 3. Append DQUOTE to output. 900 4. For each character char in input_string: 902 1. If char is "\" or DQUOTE: 904 1. Append "\" to output. 906 2. Append char to output. 908 5. Append DQUOTE to output. 910 6. Return output. 912 4.1.7. Serializing a Token 914 Given a token as input_token, return an ASCII string suitable for use 915 in a HTTP header value. 917 1. If input_token is not a sequence of characters, the first 918 character is not ALPHA or "*", or the remaining contain a 919 character not in tchar, ":" or "/", fail serialisation. 921 2. Let output be an empty string. 923 3. Append input_token to output. 925 4. Return output. 927 4.1.8. Serializing a Byte Sequence 929 Given a byte sequence as input_bytes, return an ASCII string suitable 930 for use in a HTTP header value. 932 1. If input_bytes is not a sequence of bytes, fail serialisation. 934 2. Let output be an empty string. 936 3. Append ":" to output. 938 4. Append the result of base64-encoding input_bytes as per 939 [RFC4648], Section 4, taking account of the requirements below. 941 5. Append ":" to output. 943 6. Return output. 945 The encoded data is required to be padded with "=", as per [RFC4648], 946 Section 3.2. 948 Likewise, encoded data SHOULD have pad bits set to zero, as per 949 [RFC4648], Section 3.5, unless it is not possible to do so due to 950 implementation constraints. 952 4.1.9. Serializing a Boolean 954 Given a Boolean as input_boolean, return an ASCII string suitable for 955 use in a HTTP header value. 957 1. If input_boolean is not a boolean, fail serialisation. 959 2. Let output be an empty string. 961 3. Append "?" to output. 963 4. If input_boolean is true, append "1" to output. 965 5. If input_boolean is false, append "0" to output. 967 6. Return output. 969 4.2. Parsing Header Fields into Structured Headers 971 When a receiving implementation parses HTTP header fields that are 972 known to be Structured Headers, it is important that care be taken, 973 as there are a number of edge cases that can cause interoperability 974 or even security problems. This section specifies the algorithm for 975 doing so. 977 Given an array of bytes input_bytes that represents the chosen 978 header's field-value (which is empty if that header is not present), 979 and header_type (one of "dictionary", "list", or "item"), return the 980 parsed header value. 982 1. Convert input_bytes into an ASCII string input_string; if 983 conversion fails, fail parsing. 985 2. Discard any leading SP characters from input_string. 987 3. If header_type is "list", let output be the result of running 988 Parsing a List (Section 4.2.1) with input_string. 990 4. If header_type is "dictionary", let output be the result of 991 running Parsing a Dictionary (Section 4.2.2) with input_string. 993 5. If header_type is "item", let output be the result of running 994 Parsing an Item (Section 4.2.3) with input_string. 996 6. Discard any leading SP characters from input_string. 998 7. If input_string is not empty, fail parsing. 1000 8. Otherwise, return output. 1002 When generating input_bytes, parsers MUST combine all instances of 1003 the target header field into one comma-separated field-value, as per 1004 [RFC7230], Section 3.2.2; this assures that the header is processed 1005 correctly. 1007 For Lists and Dictionaries, this has the effect of correctly 1008 concatenating all instances of the header field, as long as 1009 individual individual members of the top-level data structure are not 1010 split across multiple header instances. 1012 Strings split across multiple header instances will have 1013 unpredictable results, because comma(s) and whitespace inserted upon 1014 combination will become part of the string output by the parser. 1015 Since concatenation might be done by an upstream intermediary, the 1016 results are not under the control of the serializer or the parser. 1018 Tokens, Integers, Decimals and Byte Sequences cannot be split across 1019 multiple headers because the inserted commas will cause parsing to 1020 fail. 1022 If parsing fails - including when calling another algorithm - the 1023 entire header field's value MUST be ignored (i.e., treated as if the 1024 header field were not present in the message). This is intentionally 1025 strict, to improve interoperability and safety, and specifications 1026 referencing this document are not allowed to loosen this requirement. 1028 Note that this requirement does not apply to an implementation that 1029 is not parsing the header field; for example, an intermediary is not 1030 required to strip a failing header field from a message before 1031 forwarding it. 1033 4.2.1. Parsing a List 1035 Given an ASCII string as input_string, return an array of 1036 (item_or_inner_list, parameters) tuples. input_string is modified to 1037 remove the parsed value. 1039 1. Let members be an empty array. 1041 2. While input_string is not empty: 1043 1. Append the result of running Parsing an Item or Inner List 1044 (Section 4.2.1.1) with input_string to members. 1046 2. Discard any leading SP characters from input_string. 1048 3. If input_string is empty, return members. 1050 4. Consume the first character of input_string; if it is not 1051 COMMA, fail parsing. 1053 5. Discard any leading SP characters from input_string. 1055 6. If input_string is empty, there is a trailing comma; fail 1056 parsing. 1058 3. No structured data has been found; return members (which is 1059 empty). 1061 4.2.1.1. Parsing an Item or Inner List 1063 Given an ASCII string as input_string, return the tuple 1064 (item_or_inner_list, parameters), where item_or_inner_list can be 1065 either a single bare item, or an array of (bare_item, parameters) 1066 tuples. input_string is modified to remove the parsed value. 1068 1. If the first character of input_string is "(", return the result 1069 of running Parsing an Inner List (Section 4.2.1.2) with 1070 input_string. 1072 2. Return the result of running Parsing an Item (Section 4.2.3) with 1073 input_string. 1075 4.2.1.2. Parsing an Inner List 1077 Given an ASCII string as input_string, return the tuple (inner_list, 1078 parameters), where inner_list is an array of (bare_item, parameters) 1079 tuples. input_string is modified to remove the parsed value. 1081 1. Consume the first character of input_string; if it is not "(", 1082 fail parsing. 1084 2. Let inner_list be an empty array. 1086 3. While input_string is not empty: 1088 1. Discard any leading SP characters from input_string. 1090 2. If the first character of input_string is ")": 1092 1. Consume the first character of input_string. 1094 2. Let parameters be the result of running Parsing 1095 Parameters (Section 4.2.3.2) with input_string. 1097 3. Return the tuple (inner_list, parameters). 1099 3. Let item be the result of running Parsing an Item 1100 (Section 4.2.3) with input_string. 1102 4. Append item to inner_list. 1104 5. If the first character of input_string is not SP or ")", fail 1105 parsing. 1107 4. The end of the inner list was not found; fail parsing. 1109 4.2.2. Parsing a Dictionary 1111 Given an ASCII string as input_string, return an ordered map whose 1112 values are (item_or_inner_list, parameters) tuples. input_string is 1113 modified to remove the parsed value. 1115 1. Let dictionary be an empty, ordered map. 1117 2. While input_string is not empty: 1119 1. Let this_key be the result of running Parsing a Key 1120 (Section 4.2.3.3) with input_string. 1122 2. If the first character of input_string is "=": 1124 1. Consume the first character of input_string. 1126 2. Let member be the result of running Parsing an Item or 1127 Inner List (Section 4.2.1.1) with input_string. 1129 3. Otherwise: 1131 1. Let value be Boolean true. 1133 2. Let parameters be an empty, ordered map. 1135 3. Let member be the tuple (value, parameters). 1137 4. Add name this_key with value member to dictionary. If 1138 dictionary already contains a name this_key (comparing 1139 character-for-character), overwrite its value. 1141 5. Discard any leading SP characters from input_string. 1143 6. If input_string is empty, return dictionary. 1145 7. Consume the first character of input_string; if it is not 1146 COMMA, fail parsing. 1148 8. Discard any leading SP characters from input_string. 1150 9. If input_string is empty, there is a trailing comma; fail 1151 parsing. 1153 3. No structured data has been found; return dictionary (which is 1154 empty). 1156 4.2.3. Parsing an Item 1158 Given an ASCII string as input_string, return a (bare_item, 1159 parameters) tuple. input_string is modified to remove the parsed 1160 value. 1162 1. Let bare_item be the result of running Parsing a Bare Item 1163 (Section 4.2.3.1) with input_string. 1165 2. Let parameters be the result of running Parsing Parameters 1166 (Section 4.2.3.2) with input_string. 1168 3. Return the tuple (bare_item, parameters). 1170 4.2.3.1. Parsing a Bare Item 1172 Given an ASCII string as input_string, return a bare item. 1173 input_string is modified to remove the parsed value. 1175 1. If the first character of input_string is a "-" or a DIGIT, 1176 return the result of running Parsing a Number (Section 4.2.4) 1177 with input_string. 1179 2. If the first character of input_string is a DQUOTE, return the 1180 result of running Parsing a String (Section 4.2.5) with 1181 input_string. 1183 3. If the first character of input_string is ":", return the result 1184 of running Parsing a Byte Sequence (Section 4.2.7) with 1185 input_string. 1187 4. If the first character of input_string is "?", return the result 1188 of running Parsing a Boolean (Section 4.2.8) with input_string. 1190 5. If the first character of input_string is an ALPHA or "*", return 1191 the result of running Parsing a Token (Section 4.2.6) with 1192 input_string. 1194 6. Otherwise, the item type is unrecognized; fail parsing. 1196 4.2.3.2. Parsing Parameters 1198 Given an ASCII string as input_string, return an ordered map whose 1199 values are bare items. input_string is modified to remove the parsed 1200 value. 1202 1. Let parameters be an empty, ordered map. 1204 2. While input_string is not empty: 1206 1. If the first character of input_string is not ";", exit the 1207 loop. 1209 2. Consume a ";" character from the beginning of input_string. 1211 3. Discard any leading SP characters from input_string. 1213 4. let param_name be the result of running Parsing a Key 1214 (Section 4.2.3.3) with input_string. 1216 5. Let param_value be Boolean true. 1218 6. If the first character of input_string is "=": 1220 1. Consume the "=" character at the beginning of 1221 input_string. 1223 2. Let param_value be the result of running Parsing a Bare 1224 Item (Section 4.2.3.1) with input_string. 1226 7. Append key param_name with value param_value to parameters. 1227 If parameters already contains a name param_name (comparing 1228 character-for-character), overwrite its value. 1230 3. Return parameters. 1232 4.2.3.3. Parsing a Key 1234 Given an ASCII string as input_string, return a key. input_string is 1235 modified to remove the parsed value. 1237 1. If the first character of input_string is not lcalpha, fail 1238 parsing. 1240 2. Let output_string be an empty string. 1242 3. While input_string is not empty: 1244 1. If the first character of input_string is not one of lcalpha, 1245 DIGIT, "_", "-", ".", or "*", return output_string. 1247 2. Let char be the result of removing the first character of 1248 input_string. 1250 3. Append char to output_string. 1252 4. Return output_string. 1254 4.2.4. Parsing a Number 1256 Given an ASCII string as input_string, return a number. input_string 1257 is modified to remove the parsed value. 1259 NOTE: This algorithm parses both Integers (Section 3.3.1) and 1260 Decimals (Section 3.3.2), and returns the corresponding structure. 1262 1. Let type be "integer". 1264 2. Let sign be 1. 1266 3. Let input_number be an empty string. 1268 4. If the first character of input_string is "-", consume it and 1269 set sign to -1. 1271 5. If input_string is empty, there is an empty integer; fail 1272 parsing. 1274 6. If the first character of input_string is not a DIGIT, fail 1275 parsing. 1277 7. While input_string is not empty: 1279 1. Let char be the result of consuming the first character of 1280 input_string. 1282 2. If char is a DIGIT, append it to input_number. 1284 3. Else, if type is "integer" and char is ".": 1286 1. If input_number contains more than 12 characters, fail 1287 parsing. 1289 2. Otherwise, append char to input_number and set type to 1290 "decimal". 1292 4. Otherwise, prepend char to input_string, and exit the loop. 1294 5. If type is "integer" and input_number contains more than 15 1295 characters, fail parsing. 1297 6. If type is "decimal" and input_number contains more than 16 1298 characters, fail parsing. 1300 8. If type is "integer": 1302 1. Parse input_number as an integer and let output_number be 1303 the product of the result and sign. 1305 2. If output_number is outside the range -999,999,999,999,999 1306 to 999,999,999,999,999 inclusive, fail parsing. 1308 9. Otherwise: 1310 1. If the final character of input_number is ".", fail parsing. 1312 2. If the number of characters after "." in input_number is 1313 greater than three, fail parsing. 1315 3. Parse input_number as a decimal number and let output_number 1316 be the product of the result and sign. 1318 10. Return output_number. 1320 4.2.5. Parsing a String 1322 Given an ASCII string as input_string, return an unquoted string. 1323 input_string is modified to remove the parsed value. 1325 1. Let output_string be an empty string. 1327 2. If the first character of input_string is not DQUOTE, fail 1328 parsing. 1330 3. Discard the first character of input_string. 1332 4. While input_string is not empty: 1334 1. Let char be the result of consuming the first character of 1335 input_string. 1337 2. If char is a backslash ("\"): 1339 1. If input_string is now empty, fail parsing. 1341 2. Let next_char be the result of consuming the first 1342 character of input_string. 1344 3. If next_char is not DQUOTE or "\", fail parsing. 1346 4. Append next_char to output_string. 1348 3. Else, if char is DQUOTE, return output_string. 1350 4. Else, if char is in the range %x00-1f or %x7f (i.e., is not 1351 in VCHAR or SP), fail parsing. 1353 5. Else, append char to output_string. 1355 5. Reached the end of input_string without finding a closing DQUOTE; 1356 fail parsing. 1358 4.2.6. Parsing a Token 1360 Given an ASCII string as input_string, return a token. input_string 1361 is modified to remove the parsed value. 1363 1. If the first character of input_string is not ALPHA or "*", fail 1364 parsing. 1366 2. Let output_string be an empty string. 1368 3. While input_string is not empty: 1370 1. If the first character of input_string is not in tchar, ":" 1371 or "/", return output_string. 1373 2. Let char be the result of consuming the first character of 1374 input_string. 1376 3. Append char to output_string. 1378 4. Return output_string. 1380 4.2.7. Parsing a Byte Sequence 1382 Given an ASCII string as input_string, return a byte sequence. 1383 input_string is modified to remove the parsed value. 1385 1. If the first character of input_string is not ":", fail parsing. 1387 2. Discard the first character of input_string. 1389 3. If there is not a ":" character before the end of input_string, 1390 fail parsing. 1392 4. Let b64_content be the result of consuming content of 1393 input_string up to but not including the first instance of the 1394 character ":". 1396 5. Consume the ":" character at the beginning of input_string. 1398 6. If b64_content contains a character not included in ALPHA, DIGIT, 1399 "+", "/" and "=", fail parsing. 1401 7. Let binary_content be the result of Base 64 Decoding [RFC4648] 1402 b64_content, synthesizing padding if necessary (note the 1403 requirements about recipient behaviour below). 1405 8. Return binary_content. 1407 Because some implementations of base64 do not allow reject of encoded 1408 data that is not properly "=" padded (see [RFC4648], Section 3.2), 1409 parsers SHOULD NOT fail when it is not present, unless they cannot be 1410 configured to do so. 1412 Because some implementations of base64 do not allow rejection of 1413 encoded data that has non-zero pad bits (see [RFC4648], Section 3.5), 1414 parsers SHOULD NOT fail when it is present, unless they cannot be 1415 configured to do so. 1417 This specification does not relax the requirements in [RFC4648], 1418 Section 3.1 and 3.3; therefore, parsers MUST fail on characters 1419 outside the base64 alphabet, and on line feeds in encoded data. 1421 4.2.8. Parsing a Boolean 1423 Given an ASCII string as input_string, return a Boolean. input_string 1424 is modified to remove the parsed value. 1426 1. If the first character of input_string is not "?", fail parsing. 1428 2. Discard the first character of input_string. 1430 3. If the first character of input_string matches "1", discard the 1431 first character, and return true. 1433 4. If the first character of input_string matches "0", discard the 1434 first character, and return false. 1436 5. No value has matched; fail parsing. 1438 5. IANA Considerations 1440 This draft has no actions for IANA. 1442 6. Security Considerations 1444 The size of most types defined by Structured Headers is not limited; 1445 as a result, extremely large header fields could be an attack vector 1446 (e.g., for resource consumption). Most HTTP implementations limit 1447 the sizes of individual header fields as well as the overall header 1448 block size to mitigate such attacks. 1450 It is possible for parties with the ability to inject new HTTP header 1451 fields to change the meaning of a Structured Header. In some 1452 circumstances, this will cause parsing to fail, but it is not 1453 possible to reliably fail in all such circumstances. 1455 7. References 1457 7.1. Normative References 1459 [RFC0020] Cerf, V., "ASCII format for network interchange", STD 80, 1460 RFC 20, DOI 10.17487/RFC0020, October 1969, 1461 . 1463 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1464 Requirement Levels", BCP 14, RFC 2119, 1465 DOI 10.17487/RFC2119, March 1997, 1466 . 1468 [RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data 1469 Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006, 1470 . 1472 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 1473 Specifications: ABNF", STD 68, RFC 5234, 1474 DOI 10.17487/RFC5234, January 2008, 1475 . 1477 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 1478 Protocol (HTTP/1.1): Message Syntax and Routing", 1479 RFC 7230, DOI 10.17487/RFC7230, June 2014, 1480 . 1482 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1483 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1484 May 2017, . 1486 7.2. Informative References 1488 [IEEE754] IEEE, "IEEE Standard for Floating-Point Arithmetic", 1489 IEEE 754-2019, DOI 10.1109/IEEESTD.2019.8766229, 1490 ISBN 978-1-5044-5924-2, July 2019, 1491 . 1493 [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 1494 Protocol (HTTP/1.1): Semantics and Content", RFC 7231, 1495 DOI 10.17487/RFC7231, June 2014, 1496 . 1498 [RFC7493] Bray, T., Ed., "The I-JSON Message Format", RFC 7493, 1499 DOI 10.17487/RFC7493, March 2015, 1500 . 1502 [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext 1503 Transfer Protocol Version 2 (HTTP/2)", RFC 7540, 1504 DOI 10.17487/RFC7540, May 2015, 1505 . 1507 [RFC7541] Peon, R. and H. Ruellan, "HPACK: Header Compression for 1508 HTTP/2", RFC 7541, DOI 10.17487/RFC7541, May 2015, 1509 . 1511 [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data 1512 Interchange Format", STD 90, RFC 8259, 1513 DOI 10.17487/RFC8259, December 2017, 1514 . 1516 [UTF-8] Yergeau, F., "UTF-8, a transformation format of ISO 1517 10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November 1518 2003, . 1520 7.3. URIs 1522 [1] https://lists.w3.org/Archives/Public/ietf-http-wg/ 1524 [2] https://httpwg.github.io/ 1526 [3] https://github.com/httpwg/http-extensions/labels/header-structure 1528 [4] https://github.com/httpwg/structured-header-tests 1530 [5] https://github.com/httpwg/wiki/wiki/Structured-Headers 1532 [6] https://github.com/httpwg/structured-header-tests 1534 Appendix A. Acknowledgements 1536 Many thanks to Matthew Kerwin for his detailed feedback and careful 1537 consideration during the development of this specification. 1539 Appendix B. Frequently Asked Questions 1541 B.1. Why not JSON? 1543 Earlier proposals for structured headers were based upon JSON 1544 [RFC8259]. However, constraining its use to make it suitable for 1545 HTTP header fields required senders and recipients to implement 1546 specific additional handling. 1548 For example, JSON has specification issues around large numbers and 1549 objects with duplicate members. Although advice for avoiding these 1550 issues is available (e.g., [RFC7493]), it cannot be relied upon. 1552 Likewise, JSON strings are by default Unicode strings, which have a 1553 number of potential interoperability issues (e.g., in comparison). 1554 Although implementers can be advised to avoid non-ASCII content where 1555 unnecessary, this is difficult to enforce. 1557 Another example is JSON's ability to nest content to arbitrary 1558 depths. Since the resulting memory commitment might be unsuitable 1559 (e.g., in embedded and other limited server deployments), it's 1560 necessary to limit it in some fashion; however, existing JSON 1561 implementations have no such limits, and even if a limit is 1562 specified, it's likely that some header field definition will find a 1563 need to violate it. 1565 Because of JSON's broad adoption and implementation, it is difficult 1566 to impose such additional constraints across all implementations; 1567 some deployments would fail to enforce them, thereby harming 1568 interoperability. In short, if it looks like JSON, people will be 1569 tempted to use a JSON parser / serialiser on header fields. 1571 Since a major goal for Structured Headers is to improve 1572 interoperability and simplify implementation, these concerns led to a 1573 format that requires a dedicated parser and serializer. 1575 Additionally, there were widely shared feelings that JSON doesn't 1576 "look right" in HTTP headers. 1578 B.2. Structured Headers don't "fit" my data. 1580 Structured headers intentionally limits the complexity of data 1581 structures, to assure that it can be processed in a performant manner 1582 with little overhead. This means that work is necessary to fit some 1583 data types into them. 1585 Sometimes, this can be achieved by creating limited substructures in 1586 values, and/or using more than one header. For example, consider: 1588 Example-Thing: name="Widget", cost=89.2, descriptions=(foo bar) 1589 Example-Description: foo; url="https://example.net"; context=123, 1590 bar; url="https://example.org"; context=456 1592 Since the description contains an array of key/value pairs, we use a 1593 List to represent them, with the token for each item in the array 1594 used to identify it in the "descriptions" member of the Example-Thing 1595 dictionary header. 1597 When specifying more than one header, it's important to remember to 1598 describe what a processor's behaviour should be when one of the 1599 headers is missing. 1601 If you need to fit arbitrarily complex data into a header, Structured 1602 Headers is probably a poor fit for your use case. 1604 Appendix C. Implementation Notes 1606 A generic implementation of this specification should expose the top- 1607 level parse (Section 4.2) and serialize (Section 4.1) functions. 1608 They need not be functions; for example, it could be implemented as 1609 an object, with methods for each of the different top-level types. 1611 For interoperability, it's important that generic implementations be 1612 complete and follow the algorithms closely; see Section 1.1. To aid 1613 this, a common test suite is being maintained by the community at 1614 https://github.com/httpwg/structured-header-tests [6]. 1616 Implementers should note that dictionaries and parameters are order- 1617 preserving maps. Some headers may not convey meaning in the ordering 1618 of these data types, but it should still be exposed so that 1619 applications which need to use it will have it available. 1621 Likewise, implementations should note that it's important to preserve 1622 the distinction between tokens and strings. While most programming 1623 languages have native types that map to the other types well, it may 1624 be necessary to create a wrapper "token" object or use a parameter on 1625 functions to assure that these types remain separate. 1627 Appendix D. Changes 1629 _RFC Editor: Please remove this section before publication._ 1631 D.1. Since draft-ietf-httpbis-header-structure-14 1633 o Editorial improvements. 1635 o Allow empty dictionary values (#992). 1637 o Change value of omitted parameter value to True (#995). 1639 o Explain more about splitting dictionaries and lists across header 1640 instances (#997). 1642 o Disallow HTAB, replace OWS with spaces (#998). 1644 o Change byte sequence delimiters from "*" to ":" (#991). 1646 o Allow tokens to start with "*" (#991). 1648 o Change Floats to fixed-precision Decimals (#982). 1650 o Round the fractional component of decimal, rather than truncating 1651 it (#982). 1653 o Handle duplicate dictionary and parameter keys by overwriting 1654 their values, rather than failing (#997). 1656 o Allow "." in key (#1027). 1658 o Check first character of key in serialisation (#1037). 1660 o Talk about greasing headers (#1015). 1662 D.2. Since draft-ietf-httpbis-header-structure-13 1664 o Editorial improvements. 1666 o Define "structured header name" and "structured header value" 1667 terms (#908). 1669 o Corrected text about valid characters in strings (#931). 1671 o Removed most instances of the word "textual", as it was redundant 1672 (#915). 1674 o Allowed parameters on Items and Inner Lists (#907). 1676 o Expand the range of characters in token (#961). 1678 o Disallow OWS before ";" delimiter in parameters (#961). 1680 D.3. Since draft-ietf-httpbis-header-structure-12 1682 o Editorial improvements. 1684 o Reworked float serialisation (#896). 1686 o Don't add a trailing space in inner-list (#904). 1688 D.4. Since draft-ietf-httpbis-header-structure-11 1690 o Allow * in key (#844). 1692 o Constrain floats to six digits of precision (#848). 1694 o Allow dictionary members to have parameters (#842). 1696 D.5. Since draft-ietf-httpbis-header-structure-10 1698 o Update abstract (#799). 1700 o Input and output are now arrays of bytes (#662). 1702 o Implementations need to preserve difference between token and 1703 string (#790). 1705 o Allow empty dictionaries and lists (#781). 1707 o Change parameterized lists to have primary items (#797). 1709 o Allow inner lists in both dictionaries and lists; removes lists of 1710 lists (#816). 1712 o Subsume Parameterised Lists into Lists (#839). 1714 D.6. Since draft-ietf-httpbis-header-structure-09 1716 o Changed Boolean from T/F to 1/0 (#784). 1718 o Parameters are now ordered maps (#765). 1720 o Clamp integers to 15 digits (#737). 1722 D.7. Since draft-ietf-httpbis-header-structure-08 1724 o Disallow whitespace before items properly (#703). 1726 o Created "key" for use in dictionaries and parameters, rather than 1727 relying on identifier (#702). Identifiers have a separate minimum 1728 supported size. 1730 o Expanded the range of special characters allowed in identifier to 1731 include all of ALPHA, ".", ":", and "%" (#702). 1733 o Use "?" instead of "!" to indicate a Boolean (#719). 1735 o Added "Intentionally Strict Processing" (#684). 1737 o Gave better names for referring specs to use in Parameterised 1738 Lists (#720). 1740 o Added Lists of Lists (#721). 1742 o Rename Identifier to Token (#725). 1744 o Add implementation guidance (#727). 1746 D.8. Since draft-ietf-httpbis-header-structure-07 1748 o Make Dictionaries ordered mappings (#659). 1750 o Changed "binary content" to "byte sequence" to align with Infra 1751 specification (#671). 1753 o Changed "mapping" to "map" for #671. 1755 o Don't fail if byte sequences aren't "=" padded (#658). 1757 o Add Booleans (#683). 1759 o Allow identifiers in items again (#629). 1761 o Disallowed whitespace before items (#703). 1763 o Explain the consequences of splitting a string across multiple 1764 headers (#686). 1766 D.9. Since draft-ietf-httpbis-header-structure-06 1768 o Add a FAQ. 1770 o Allow non-zero pad bits. 1772 o Explicitly check for integers that violate constraints. 1774 D.10. Since draft-ietf-httpbis-header-structure-05 1776 o Reorganise specification to separate parsing out. 1778 o Allow referencing specs to use ABNF. 1780 o Define serialisation algorithms. 1782 o Refine relationship between ABNF, parsing and serialisation 1783 algorithms. 1785 D.11. Since draft-ietf-httpbis-header-structure-04 1787 o Remove identifiers from item. 1789 o Remove most limits on sizes. 1791 o Refine number parsing. 1793 D.12. Since draft-ietf-httpbis-header-structure-03 1795 o Strengthen language around failure handling. 1797 D.13. Since draft-ietf-httpbis-header-structure-02 1799 o Split Numbers into Integers and Floats. 1801 o Define number parsing. 1803 o Tighten up binary parsing and give it an explicit end delimiter. 1805 o Clarify that mappings are unordered. 1807 o Allow zero-length strings. 1809 o Improve string parsing algorithm. 1811 o Improve limits in algorithms. 1813 o Require parsers to combine header fields before processing. 1815 o Throw an error on trailing garbage. 1817 D.14. Since draft-ietf-httpbis-header-structure-01 1819 o Replaced with draft-nottingham-structured-headers. 1821 D.15. Since draft-ietf-httpbis-header-structure-00 1823 o Added signed 64bit integer type. 1825 o Drop UTF8, and settle on BCP137 ::EmbeddedUnicodeChar for h1- 1826 unicode-string. 1828 o Change h1_blob delimiter to ":" since "'" is valid t_char 1830 Authors' Addresses 1832 Mark Nottingham 1833 Fastly 1835 Email: mnot@mnot.net 1836 URI: https://www.mnot.net/ 1838 Poul-Henning Kamp 1839 The Varnish Cache Project 1841 Email: phk@varnish-cache.org