idnits 2.17.1 draft-reschke-http-jfv-12.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == There is 1 instance of lines with non-ascii characters in the document. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 287: '... MUST NOT use duplicate object names...' Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (September 1, 2020) is 1331 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'CLEARSITE' is mentioned on line 420, but not defined == Missing Reference: 'FEATUREPOL' is mentioned on line 425, but not defined == Missing Reference: 'REPORTING' is mentioned on line 412, but not defined == Outdated reference: A later version (-19) exists of draft-ietf-httpbis-semantics-11 Summary: 2 errors (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. F. Reschke 3 Internet-Draft greenbytes 4 Intended status: Informational September 1, 2020 5 Expires: March 5, 2021 7 A JSON Encoding for HTTP Field Values 8 draft-reschke-http-jfv-12 10 Abstract 12 This document establishes a convention for use of JSON-encoded field 13 values in HTTP fields. 15 Editorial Note 17 This note is to be removed before publishing as an RFC. 19 Distribution of this document is unlimited. Although this is not a 20 work item of the HTTPbis Working Group, comments should be sent to 21 the Hypertext Transfer Protocol (HTTP) mailing list at ietf-http- 22 wg@w3.org (mailto:ietf-http-wg@w3.org), which may be joined by 23 sending a message with subject "subscribe" to ietf-http-wg- 24 request@w3.org (mailto:ietf-http-wg- 25 request@w3.org?subject=subscribe). 27 Discussions of the HTTPbis Working Group are archived at 28 . 30 XML versions and latest edits for this document are available from 31 . 33 The changes in this draft are summarized in Appendix B.15. 35 Status of This Memo 37 This Internet-Draft is submitted in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF). Note that other groups may also distribute 42 working documents as Internet-Drafts. The list of current Internet- 43 Drafts is at https://datatracker.ietf.org/drafts/current/. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 49 This Internet-Draft will expire on March 5, 2021. 51 Copyright Notice 53 Copyright (c) 2020 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 58 license-info) in effect on the date of publication of this document. 59 Please review these documents carefully, as they describe your rights 60 and restrictions with respect to this document. Code Components 61 extracted from this document must include Simplified BSD License text 62 as described in Section 4.e of the Trust Legal Provisions and are 63 provided without warranty as described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 68 2. Data Model and Format . . . . . . . . . . . . . . . . . . . . 4 69 3. Sender Requirements . . . . . . . . . . . . . . . . . . . . . 5 70 4. Recipient Requirements . . . . . . . . . . . . . . . . . . . 5 71 5. Using this Format in Field Definitions . . . . . . . . . . . 5 72 6. Deployment Considerations . . . . . . . . . . . . . . . . . . 6 73 7. Interoperability Considerations . . . . . . . . . . . . . . . 6 74 7.1. Encoding and Characters . . . . . . . . . . . . . . . . . 6 75 7.2. Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 6 76 7.3. Object Constraints . . . . . . . . . . . . . . . . . . . 7 77 8. Internationalization Considerations . . . . . . . . . . . . . 7 78 9. Security Considerations . . . . . . . . . . . . . . . . . . . 7 79 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 7 80 10.1. Normative References . . . . . . . . . . . . . . . . . . 7 81 10.2. Informative References . . . . . . . . . . . . . . . . . 8 82 10.3. Specifications Using This Syntax (at some point of 83 time) . . . . . . . . . . . . . . . . . . . . . . . . . 9 84 Appendix A. Use of JSON Field Value Encoding in the Wild . . . . 9 85 A.1. W3C Reporting API Specification . . . . . . . . . . . . . 9 86 A.2. W3C Clear Site Data Specification . . . . . . . . . . . . 9 87 A.3. W3C Feature Policy Specification . . . . . . . . . . . . 10 88 Appendix B. Change Log . . . . . . . . . . . . . . . . . . . . . 10 89 B.1. Since draft-reschke-http-jfv-00 . . . . . . . . . . . . . 10 90 B.2. Since draft-reschke-http-jfv-01 . . . . . . . . . . . . . 10 91 B.3. Since draft-reschke-http-jfv-02 . . . . . . . . . . . . . 10 92 B.4. Since draft-reschke-http-jfv-03 . . . . . . . . . . . . . 10 93 B.5. Since draft-reschke-http-jfv-04 . . . . . . . . . . . . . 10 94 B.6. Since draft-ietf-httpbis-jfv-00 . . . . . . . . . . . . . 10 95 B.7. Since draft-ietf-httpbis-jfv-01 . . . . . . . . . . . . . 11 96 B.8. Since draft-ietf-httpbis-jfv-02 . . . . . . . . . . . . . 11 97 B.9. Since draft-reschke-http-jfv-05 . . . . . . . . . . . . . 11 98 B.10. Since draft-reschke-http-jfv-06 . . . . . . . . . . . . . 11 99 B.11. Since draft-reschke-http-jfv-07 . . . . . . . . . . . . . 11 100 B.12. Since draft-reschke-http-jfv-08 . . . . . . . . . . . . . 11 101 B.13. Since draft-reschke-http-jfv-09 . . . . . . . . . . . . . 11 102 B.14. Since draft-reschke-http-jfv-10 . . . . . . . . . . . . . 12 103 B.15. Since draft-reschke-http-jfv-11 . . . . . . . . . . . . . 12 104 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 12 105 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 12 107 1. Introduction 109 Defining syntax for new HTTP fields ([HTTP], Section 5) is non- 110 trivial. Among the commonly encountered problems are: 112 o There is no common syntax for complex field values. Several well- 113 known fields do use a similarly looking syntax, but it is hard to 114 write generic parsing code that will both correctly handle valid 115 field values but also reject invalid ones. 117 o The HTTP message format allows fields to repeat, so field syntax 118 needs to be designed in a way that these cases are either 119 meaningful, or can be unambiguously detected and rejected. 121 o HTTP does not define a character encoding scheme ([RFC6365], 122 Section 2), so fields are either stuck with US-ASCII ([RFC0020]), 123 or need out-of-band information to decide what encoding scheme is 124 used. Furthermore, APIs usually assume a default encoding scheme 125 in order to map from octet sequences to strings (for instance, 126 [XMLHttpRequest] uses the IDL type "ByteString", effectively 127 resulting in the ISO-8859-1 character encoding scheme [ISO-8859-1] 128 being used). 130 (See Section 5.7 of [HTTP] for a summary of considerations for new 131 fields.) 133 This specification addresses the issues listed above by defining both 134 a generic JSON-based ([RFC8259]) data model and a concrete wire 135 format that can be used in definitions of new fields, where the goals 136 were: 138 o to be compatible with field recombination when fields occur 139 multiple times in a single message (Section 5.1 of [HTTP]), and 141 o not to use any problematic characters in the field value (non- 142 ASCII characters and certain whitespace characters). 144 | *Note:* [HSTRUCT], a work item of the IETF HTTP Working Group, 145 | is a different attempt to address this set of problems - it 146 | tries to identify and formalize common field structures in 147 | existing fields; the syntax defined over there would usually 148 | lead to a more compact notation. 150 2. Data Model and Format 152 In HTTP, fields with the same field name can occur multiple times 153 within a single message (Section 5.1 of [HTTP]). When this happens, 154 recipients are allowed to combine the field values using commas as 155 delimiter. This rule matches nicely JSON's array format (Section 5 156 of [RFC8259]). Thus, the basic data model used here is the JSON 157 array. 159 Field definitions that need only a single value can restrict 160 themselves to arrays of length 1, and are encouraged to define error 161 handling in case more values are received (such as "first wins", 162 "last wins", or "abort with fatal error message"). 164 JSON arrays are mapped to field values by creating a sequence of 165 serialized member elements, separated by commas and optionally 166 whitespace. This is equivalent to using the full JSON array format, 167 while leaving out the "begin-array" ('[') and "end-array" (']') 168 delimiters. 170 The ABNF character names and classes below are used (copied from 171 [RFC5234], Appendix B.1): 173 CR = %x0D ; carriage return 174 HTAB = %x09 ; horizontal tab 175 LF = %x0A ; line feed 176 SP = %x20 ; space 177 VCHAR = %x21-7E ; visible (printing) characters 179 Characters in JSON strings that are not allowed or discouraged in 180 HTTP field values - that is, not in the "VCHAR" definition - need to 181 be represented using JSON's "backslash" escaping mechanism 182 ([RFC8259], Section 7). 184 The control characters CR, LF, and HTAB do not appear inside JSON 185 strings, but can be used outside (line breaks, indentation etc.). 186 These characters need to be either stripped or replaced by space 187 characters (ABNF "SP"). 189 Formally, using the HTTP specification's ABNF extensions defined in 190 Section 5.5 of [HTTP]: 192 json-field-value = #json-field-item 193 json-field-item = JSON-Text 194 ; see [RFC8259], Section 2, 195 ; post-processed so that only VCHAR characters 196 ; are used 198 3. Sender Requirements 200 To map a JSON array to an HTTP field value, process each array 201 element separately by: 203 1. generating the JSON representation, 205 2. stripping all JSON control characters (CR, HTAB, LF), or 206 replacing them by space ("SP") characters, 208 3. replacing all remaining non-VSPACE characters by the equivalent 209 backslash-escape sequence ([RFC8259], Section 7). 211 The resulting list of strings is transformed into an HTTP field value 212 by combining them using comma (%x2C) plus optional SP as delimiter, 213 and encoding the resulting string into an octet sequence using the 214 US-ASCII character encoding scheme ([RFC0020]). 216 4. Recipient Requirements 218 To map a set of HTTP field instances to a JSON array: 220 1. combine all field instances into a single field as per 221 Section 5.1 of [HTTP], 223 2. add a leading begin-array ("[") octet and a trailing end-array 224 ("]") octet, then 226 3. run the resulting octet sequence through a JSON parser. 228 The result of the parsing operation is either an error (in which case 229 the field values needs to be considered invalid), or a JSON array. 231 5. Using this Format in Field Definitions 233 Specifications defining new HTTP fields need to take the 234 considerations listed in Section 5.7 of [HTTP] into account. Many of 235 these will already be accounted for by using the format defined in 236 this specification. 238 Readers of HTTP-related specifications frequently expect an ABNF 239 definition of the field value syntax. This is not really needed 240 here, as the actual syntax is JSON text, as defined in Section 2 of 241 [RFC8259]. 243 A very simple way to use this JSON encoding thus is just to cite this 244 specification - specifically the "json-field-value" ABNF production 245 defined in Section 2 - and otherwise not to talk about the details of 246 the field syntax at all. 248 An alternative approach is just to repeat the ABNF-related parts from 249 Section 2. 251 This frees the specification from defining the concrete on-the-wire 252 syntax. What's left is defining the field value in terms of a JSON 253 array. An important aspect is the question of extensibility, e.g. 254 how recipients ought to treat unknown field names. In general, a 255 "must ignore" approach will allow protocols to evolve without 256 versioning or even using entire new field names. 258 6. Deployment Considerations 260 This JSON-based syntax will only apply to newly introduced fields, 261 thus backwards compatibility is not a problem. That being said, it 262 is conceivable that there is existing code that might trip over 263 double quotes not being used for HTTP's quoted-string syntax 264 (Section 5.4.1 of [HTTP]). 266 7. Interoperability Considerations 268 The "I-JSON Message Format" specification ([RFC7493]) addresses known 269 JSON interoperability pain points. This specification borrows from 270 the requirements made over there: 272 7.1. Encoding and Characters 274 This specification requires that field values use only US-ASCII 275 characters, and thus by definition use a subset of UTF-8 (Section 2.1 276 of [RFC7493]). 278 7.2. Numbers 280 Be aware of the issues around number precision, as discussed in 281 Section 2.2 of [RFC7493]. 283 7.3. Object Constraints 285 As described in Section 4 of [RFC8259], JSON parser implementations 286 differ in the handling of duplicate object names. Therefore, senders 287 MUST NOT use duplicate object names, and recipients SHOULD either 288 treat field values with duplicate names as invalid (consistent with 289 [RFC7493], Section 2.3) or use the lexically last value (consistent 290 with [ECMA-262], Section 24.3.1.1). 292 Furthermore, ordering of object members is not significant and can 293 not be relied upon. 295 8. Internationalization Considerations 297 In current versions of HTTP, field values are represented by octet 298 sequences, usually used to transmit ASCII characters, with 299 restrictions on the use of certain control characters, and no 300 associated default character encoding, nor a way to describe it 301 ([HTTP], Section 5). HTTP/2 does not change this. 303 This specification maps all characters which can cause problems to 304 JSON escape sequences, thereby solving the HTTP field 305 internationalization problem. 307 Future specifications of HTTP might change to allow non-ASCII 308 characters natively. In that case, fields using the syntax defined 309 by this specification would have a simple migration path (by just 310 stopping to require escaping of non-ASCII characters). 312 9. Security Considerations 314 Using JSON-shaped field values is believed to not introduce any new 315 threads beyond those described in Section 12 of [RFC8259], namely the 316 risk of recipients using the wrong tools to parse them. 318 Other than that, any syntax that makes extensions easy can be used to 319 smuggle information through field values; however, this concern is 320 shared with other widely used formats, such as those using parameters 321 in the form of name/value pairs. 323 10. References 325 10.1. Normative References 327 [HTTP] Fielding, R., Ed., Nottingham, M., Ed., and J. F. Reschke, 328 Ed., "HTTP Semantics", Work in Progress, Internet-Draft, 329 draft-ietf-httpbis-semantics-11, August 27, 2020, 330 . 333 [RFC0020] Cerf, V., "ASCII format for network interchange", STD 80, 334 RFC 20, DOI 10.17487/RFC0020, October 1969, 335 . 337 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 338 Specifications: ABNF", STD 68, RFC 5234, 339 DOI 10.17487/RFC5234, January 2008, 340 . 342 [RFC7493] Bray, T., Ed., "The I-JSON Message Format", RFC 7493, 343 DOI 10.17487/RFC7493, March 2015, 344 . 346 [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data 347 Interchange Format", RFC 8259, DOI 10.17487/RFC8259, 348 December 2017, . 350 10.2. Informative References 352 [ECMA-262] Ecma International, "ECMA-262 6th Edition, The ECMAScript 353 2015 Language Specification", Standard ECMA-262, June 354 2015, . 356 [HSTRUCT] Nottingham, M. and P-H. Kamp, "Structured Field Values for 357 HTTP", Work in Progress, Internet-Draft, draft-ietf- 358 httpbis-header-structure-19, June 2020, 359 . 362 [ISO-8859-1] 363 International Organization for Standardization, 364 "Information technology -- 8-bit single-byte coded graphic 365 character sets -- Part 1: Latin alphabet No. 1", ISO/ 366 IEC 8859-1:1998, 1998. 368 [RFC6365] Hoffman, P. and J. Klensin, "Terminology Used in 369 Internationalization in the IETF", BCP 166, RFC 6365, 370 DOI 10.17487/RFC6365, September 2011, 371 . 373 [XMLHttpRequest] 374 WhatWG, "XMLHttpRequest", . 376 10.3. Specifications Using This Syntax (at some point of time) 378 [CLEARSITE] 379 West, M., "Clear Site Data", W3C Working Draft WD-clear- 380 site-data-20171130, November 30, 2017, 381 . 382 Latest version available at . 385 [FEATUREPOL] 386 Clelland, I., "Feature Policy", W3C Editor's Draft , 387 . 389 [REPORTING] 390 Creager, D., Grigorik, I., Meyer, P., and M. West, 391 "Reporting API", W3C Working Draft WD-reporting- 392 1-20180925, September 25, 2018, 393 . 394 Latest version available at . 397 Appendix A. Use of JSON Field Value Encoding in the Wild 399 This section is to be removed before publishing as an RFC. 401 Since work started on this document, various specifications have 402 adopted this format. At least one of these moved away after the HTTP 403 Working Group decided to focus on [HSTRUCT] (see thread starting at 404 ). 407 The sections below summarize the current usage of this format. 409 A.1. W3C Reporting API Specification 411 Defined in W3C Working Draft "Reporting API" (Section 3.1 of 412 [REPORTING]). Still in use in latest working draft dated September 413 2018. 415 A.2. W3C Clear Site Data Specification 417 Used in earlier versions of "Clear Site Data". The current version 418 replaces the use of JSON with a custom syntax that happens to be 419 somewhat compatible with an array of JSON strings (see Section 3.1 of 420 [CLEARSITE] and for feedback). 423 A.3. W3C Feature Policy Specification 425 Originally defined in W3C document "Feature Policy" ([FEATUREPOL]), 426 but switched to use of Structured Header Fields ([HSTRUCT]). 428 Appendix B. Change Log 430 This section is to be removed before publishing as an RFC. 432 B.1. Since draft-reschke-http-jfv-00 434 Editorial fixes + working on the TODOs. 436 B.2. Since draft-reschke-http-jfv-01 438 Mention slightly increased risk of smuggling information in header 439 field values. 441 B.3. Since draft-reschke-http-jfv-02 443 Mention Kazuho Oku's proposal for abbreviated forms. 445 Added a bit of text about the motivation for a concrete JSON subset 446 (ack Cory Benfield). 448 Expand I18N section. 450 B.4. Since draft-reschke-http-jfv-03 452 Mention relation to KEY header field. 454 B.5. Since draft-reschke-http-jfv-04 456 Between June and December 2016, this was a work item of the HTTP 457 working group (see ). Work (if any) continues now on 459 . 461 Changes made while this was a work item of the HTTP Working Group: 463 B.6. Since draft-ietf-httpbis-jfv-00 465 Added example for "Accept-Encoding" (inspired by Kazuho's feedback), 466 showing a potential way to optimize the format when default values 467 apply. 469 B.7. Since draft-ietf-httpbis-jfv-01 471 Add interop discussion, building on I-JSON and ECMA-262 (see 472 ). 474 B.8. Since draft-ietf-httpbis-jfv-02 476 Move non-essential parts into appendix. 478 Updated XHR reference. 480 B.9. Since draft-reschke-http-jfv-05 482 Add meat to "Using this Format in Header Field Definitions". 484 Add a few lines on the relation to "Key". 486 Summarize current use of the format. 488 B.10. Since draft-reschke-http-jfv-06 490 RFC 5987 is obsoleted by RFC 8187. 492 Update CLEARSITE comment. 494 B.11. Since draft-reschke-http-jfv-07 496 Update JSON and HSTRUCT references. 498 FEATUREPOL doesn't use JSON syntax anymore. 500 B.12. Since draft-reschke-http-jfv-08 502 Update HSTRUCT reference. 504 Update notes about CLEARSITE and FEATUREPOL. 506 B.13. Since draft-reschke-http-jfv-09 508 Update HSTRUCT and FEATUREPOL references. 510 Update note about REPORTING. 512 Changed category to "informational". 514 B.14. Since draft-reschke-http-jfv-10 516 Update HSTRUCT reference. 518 B.15. Since draft-reschke-http-jfv-11 520 Update HSTRUCT reference. 522 Update note about FEATUREPOL (now using Structured Fields). 524 Reference [HTTP] instead if RFC723* and adjust (header) field 525 terminology accordingly. 527 Remove discussion about the relation to KEY (as that spec is dormant: 528 ). 530 Remove appendices "Examples" and "Discussion". 532 Mark "Use of JSON Field Value Encoding in the Wild" for removal in 533 RFC. 535 Acknowledgements 537 Thanks go to the Hypertext Transfer Protocol Working Group 538 participants. 540 Author's Address 542 Julian F. Reschke 543 greenbytes GmbH 544 Hafenweg 16 545 48155 Münster 546 Germany 548 Email: julian.reschke@greenbytes.de 549 URI: http://greenbytes.de/tech/webdav/