idnits 2.17.1 draft-reschke-rfc2231-in-http-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Sep 2009 rather than the newer Notice from 28 Dec 2009. (See https://trustee.ietf.org/license-info/) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([2], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (December 27, 2009) is 5234 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'ISO-8859-1' ** Obsolete normative reference: RFC 2616 (Obsoleted by RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235) Summary: 3 errors (**), 0 flaws (~~), 1 warning (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Reschke 3 Internet-Draft greenbytes 4 Intended status: Standards Track December 27, 2009 5 Expires: June 30, 2010 7 Application of RFC 2231 Encoding to 8 Hypertext Transfer Protocol (HTTP) Header Fields 9 draft-reschke-rfc2231-in-http-07 11 Abstract 13 By default, message header field parameters in Hypertext Transfer 14 Protocol (HTTP) messages can not carry characters outside the ISO- 15 8859-1 character set. RFC 2231 defines an escaping mechanism for use 16 in Multipurpose Internet Mail Extensions (MIME) headers. This 17 document specifies a profile of that encoding suitable for use in 18 HTTP. 20 Editorial Note (To be removed by RFC Editor before publication) 22 There are multiple HTTP header fields that already use RFC 2231 23 encoding in practice (Content-Disposition) or might use it in the 24 future (Link). The purpose of this document is to provide a single 25 place where the generic aspects of RFC 2231 encoding in HTTP header 26 fields is defined. 28 Distribution of this document is unlimited. Although this is not a 29 work item of the HTTPbis Working Group, comments should be sent to 30 the Hypertext Transfer Protocol (HTTP) mailing list at 31 ietf-http-wg@w3.org [1], which may be joined by sending a message 32 with subject "subscribe" to ietf-http-wg-request@w3.org [2]. 34 Discussions of the HTTPbis Working Group are archived at 35 . 37 XML versions, latest edits and the issues list for this document are 38 available from 39 . A 40 collection of test cases is available at 41 . 43 Status of this Memo 45 This Internet-Draft is submitted to IETF in full conformance with the 46 provisions of BCP 78 and BCP 79. 48 Internet-Drafts are working documents of the Internet Engineering 49 Task Force (IETF), its areas, and its working groups. Note that 50 other groups may also distribute working documents as Internet- 51 Drafts. 53 Internet-Drafts are draft documents valid for a maximum of six months 54 and may be updated, replaced, or obsoleted by other documents at any 55 time. It is inappropriate to use Internet-Drafts as reference 56 material or to cite them other than as "work in progress." 58 The list of current Internet-Drafts can be accessed at 59 http://www.ietf.org/ietf/1id-abstracts.txt. 61 The list of Internet-Draft Shadow Directories can be accessed at 62 http://www.ietf.org/shadow.html. 64 This Internet-Draft will expire on June 30, 2010. 66 Copyright Notice 68 Copyright (c) 2009 IETF Trust and the persons identified as the 69 document authors. All rights reserved. 71 This document is subject to BCP 78 and the IETF Trust's Legal 72 Provisions Relating to IETF Documents 73 (http://trustee.ietf.org/license-info) in effect on the date of 74 publication of this document. Please review these documents 75 carefully, as they describe your rights and restrictions with respect 76 to this document. Code Components extracted from this document must 77 include Simplified BSD License text as described in Section 4.e of 78 the Trust Legal Provisions and are provided without warranty as 79 described in the BSD License. 81 Table of Contents 83 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 84 2. Notational Conventions . . . . . . . . . . . . . . . . . . . . 4 85 3. A Profile of RFC 2231 for Use in HTTP . . . . . . . . . . . . 4 86 3.1. Parameter Continuations . . . . . . . . . . . . . . . . . 4 87 3.2. Parameter Value Character Set and Language Information . . 5 88 3.2.1. Examples . . . . . . . . . . . . . . . . . . . . . . . 7 89 3.3. Language specification in Encoded Words . . . . . . . . . 8 90 4. Guidelines for Usage in HTTP Header Field Definitions . . . . 8 91 4.1. When to Use the Extension . . . . . . . . . . . . . . . . 8 92 4.2. Error Handling . . . . . . . . . . . . . . . . . . . . . . 9 93 4.3. Using Multiple Instances for Internationalization . . . . 9 94 5. Security Considerations . . . . . . . . . . . . . . . . . . . 10 95 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 96 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 10 97 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 10 98 8.1. Normative References . . . . . . . . . . . . . . . . . . . 10 99 8.2. Informative References . . . . . . . . . . . . . . . . . . 10 100 Appendix A. Document History and Future Plans (to be removed 101 by RFC Editor before publication) . . . . . . . . . . 11 102 Appendix B. Change Log (to be removed by RFC Editor before 103 publication) . . . . . . . . . . . . . . . . . . . . 12 104 B.1. Since draft-reschke-rfc2231-in-http-00 . . . . . . . . . . 12 105 B.2. Since draft-reschke-rfc2231-in-http-01 . . . . . . . . . . 12 106 B.3. Since draft-reschke-rfc2231-in-http-02 . . . . . . . . . . 12 107 B.4. Since draft-reschke-rfc2231-in-http-03 . . . . . . . . . . 12 108 B.5. Since draft-reschke-rfc2231-in-http-04 . . . . . . . . . . 12 109 B.6. Since draft-reschke-rfc2231-in-http-05 . . . . . . . . . . 12 110 B.7. Since draft-reschke-rfc2231-in-http-06 . . . . . . . . . . 12 111 Appendix C. Open issues (to be removed by RFC Editor prior to 112 publication) . . . . . . . . . . . . . . . . . . . . 13 113 C.1. edit . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 114 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 13 116 1. Introduction 118 By default, message header field parameters in HTTP ([RFC2616]) 119 messages can not carry characters outside the ISO-8859-1 character 120 set ([ISO-8859-1]). RFC 2231 ([RFC2231]) defines an escaping 121 mechanism for use in MIME headers. This document specifies a profile 122 of that encoding for use in HTTP. 124 2. Notational Conventions 126 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 127 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 128 document are to be interpreted as described in [RFC2119]. 130 This specification uses the ABNF (Augmented Backus-Naur Form) 131 notation defined in [RFC5234]. The following core rules are included 132 by reference, as defined in [RFC5234], Appendix B.1: ALPHA (letters), 133 DIGIT (decimal 0-9), HEXDIG (hexadecimal 0-9/A-F/a-f) and LWSP 134 (linear white space). 136 Note that this specification uses the term "character set" for 137 consistency with other IETF specifications such as RFC 2277 (see 138 [RFC2277], Section 3). A more accurate term would be "character 139 encoding" (a mapping of code points to octet sequences). 141 3. A Profile of RFC 2231 for Use in HTTP 143 RFC 2231 defines several extensions to MIME. The sections below 144 discuss if and how they apply to HTTP. 146 In short: 148 o Parameter Continuations aren't needed (Section 3.1), 150 o Character Set and Language Information are useful, therefore a 151 simple subset is specified (Section 3.2), and 153 o Language Specifications in Encoded Words aren't needed 154 (Section 3.3). 156 3.1. Parameter Continuations 158 Section 3 of [RFC2231] defines a mechanism that deals with the length 159 limitations that apply to MIME headers. These limitations do not 160 apply to HTTP ([RFC2616], Section 19.4.7). 162 Thus in HTTP, senders MUST NOT use parameter continuations, and 163 therefore recipients do not need to support them. 165 3.2. Parameter Value Character Set and Language Information 167 Section 4 of [RFC2231] specifies how to embed language information 168 into parameter values, and also how to encode non-ASCII characters, 169 dealing with restrictions both in MIME and HTTP header parameters. 171 However, RFC 2231 does not specify a mandatory-to-implement character 172 set, making it hard for senders to decide which character set to use. 173 Thus, recipients implementing this specification MUST support the 174 character sets "ISO-8859-1" [ISO-8859-1] and "UTF-8" [RFC3629]. 176 Furthermore, RFC 2231 allows leaving out the character set 177 information. The profile defined by this specification does not 178 allow that. 180 The syntax for parameters is defined in Section 3.6 of [RFC2616] 181 (with RFC 2616 implied LWS translated to RFC 5234 LWSP): 183 parameter = attribute LWSP "=" LWSP value 185 attribute = token 186 value = token / quoted-string 188 quoted-string = 189 token = 191 This specification extends the grammar to: 193 parameter = reg-parameter / ext-parameter 195 reg-parameter = attribute LWSP "=" LWSP value 197 ext-parameter = attribute "*" LWSP "=" LWSP ext-value 199 ext-value = charset "'" [ language ] "'" value-chars 200 ; extended-initial-value, 201 ; defined in [RFC2231], Section 7 203 charset = "UTF-8" / "ISO-8859-1" / mime-charset 205 mime-charset = 1*mime-charsetc 206 mime-charsetc = ALPHA / DIGIT 207 / "!" / "#" / "$" / "%" / "&" 208 / "+" / "-" / "^" / "_" / "`" 209 / "{" / "}" / "~" 210 ; as in Section 2.3 of [RFC2978] 211 ; except that the single quote is not included 213 language = 215 value-chars = *( pct-encoded / attr-char ) 217 pct-encoded = "%" HEXDIG HEXDIG 218 ; see [RFC3986], Section 2.1 220 attr-char = ALPHA / DIGIT 221 / "-" / "." / "_" / "~" / ":" 222 / "!" / "$" / "&" / "+" 224 Thus, a parameter is either regular parameter (reg-parameter), as 225 previously defined in Section 3.6 of [RFC2616], or an extended 226 parameter (ext-parameter). 228 Extended parameters are those where the left hand side of the 229 assignment ends with an asterisk character. 231 The value part of an extended parameter (ext-value) is a token that 232 consists of three parts: the REQUIRED character set name (charset), 233 the OPTIONAL language information (language), and a character 234 sequence representing the actual value (value-chars), separated by 235 single quote characters. Note that both character set names and 236 language tags are restricted to the US-ASCII character set, and are 237 matched case-insensitively (see [RFC2978], Section 2.3 and [RFC5646], 238 Section 2.1.1). 240 Inside the value part, characters not contained in attr-char are 241 encoded into an octet sequence using the specified character set. 242 That octet sequence then is percent-encoded as specified in Section 243 2.1 of [RFC3986]. 245 Producers MUST NOT use character sets other than "UTF-8" ([RFC3629]) 246 or "ISO-8859-1" ([ISO-8859-1]). Extension character sets (ext- 247 charset) are reserved for future use. 249 Note: recipients should be prepared to handle encoding errors, 250 such as malformed or incomplete percent escape sequences, or non- 251 decodable octet sequences, in a robust manner. This specification 252 does not mandate any specific behavior, for instance the following 253 strategies are all acceptable: 255 * ignoring the parameter, 257 * stripping a non-decodable octet sequence, 259 * substituting a non-decodable octet sequence by a replacement 260 character, such as the Unicode character U+FFFD (Replacement 261 Character). 263 Note: the ABNF defined here differs from the one in 264 Section 2.3 of [RFC2978] in that it does not allow the single 265 quote character (see also RFC Editor Errata ID 1912 [3]). In 266 practice, no character set names using that character have been 267 registered at the time of this writing. 269 3.2.1. Examples 271 Non-extended notation, using "token": 273 foo: bar; title=Economy 275 Non-extended notation, using "quoted-string": 277 foo: bar; title="US-$ rates" 279 Extended notation, using the unicode character U+00A3 (POUND SIGN): 281 foo: bar; title*=iso-8859-1'en'%A3%20rates 283 Note: the Unicode pound sign character U+00A3 was encoded using ISO- 284 8859-1 into the single octet A3, then percent-encoded. Also note 285 that the space character was encoded as %20, as it is not contained 286 in attr-char. 288 Extended notation, using the unicode characters U+00A3 (POUND SIGN) 289 and U+20AC (EURO SIGN): 291 foo: bar; title*=UTF-8''%c2%a3%20and%20%e2%82%ac%20rates 293 Note: the unicode pound sign character U+00A3 was encoded using UTF-8 294 into the octet sequence C2 A3, then percent-encoded. Likewise, the 295 unicode euro sign character U+20AC was encoded into the octet 296 sequence E2 82 AC, then percent-encoded. Also note that HEXDIG 297 allows both lower-case and upper-case character, so recipients must 298 understand both, and that the language information is optional, while 299 the character set is not. 301 3.3. Language specification in Encoded Words 303 Section 5 of [RFC2231] extends the encoding defined in [RFC2047] to 304 also support language specification in encoded words. Although the 305 HTTP/1.1 specification does refer to RFC 2047 ([RFC2616], Section 306 2.2), it's not clear to which header field exactly it applies, and 307 whether it is implemented in practice (see 308 for details). 310 Thus, the RFC 2231 profile defined by this specification does not 311 include this feature. 313 4. Guidelines for Usage in HTTP Header Field Definitions 315 Specifications of HTTP header fields that use the extensions defined 316 in Section 3.2 should clearly state that. A simple way to achieve 317 this is to normatively reference this specification, and to include 318 the ext-value production into the ABNF for that header field. 320 For instance: 322 foo-header = "foo" LWSP ":" LWSP token ";" LWSP title-param 323 title-param = "title" LWSP "=" LWSP value 324 / "title*" LWSP "=" LWSP ext-value 325 ext-value = 327 [[rfcno: Note to RFC Editor: in the figure above, please replace 328 "xxxx" by the RFC number assigned to this specification.]] 330 4.1. When to Use the Extension 332 Section 4.2 of [RFC2277] requires that protocol elements containing 333 text can carry language information. Thus, the ext-value production 334 should always be used when the parameter value is of textual nature. 336 Furthermore, the extension should also be used whenever the parameter 337 value needs to carry characters not present in the US-ASCII 338 ([USASCII]) character set (note that it would be unacceptable to 339 define a new parameter that would be restricted to a subset of the 340 Unicode character set). 342 4.2. Error Handling 344 Header specifications that include parameters should also specify 345 whether same-named parameters can occur multiple times. If 346 repetitions are not allowed (and this is believed to be the common 347 case), the specification should state whether regular or the extended 348 syntax takes precedence. In the latter case, this could be used by 349 producers to use both formats without breaking recipients that do not 350 understand the syntax. 352 Example: 354 foo: bar; title="EURO exchange rates"; 355 title*=utf-8''%e2%82%ac%20exchange%20rates 357 In this case, the sender provides an ASCII version of the title for 358 legacy recipients, but also includes an internationalized version for 359 recipients understanding this specification -- the latter obviously 360 should prefer the new syntax over the old one. 362 Note: at the time of this writing, many implementations failed to 363 ignore the form they do not understand, or prioritize the ASCII 364 form although the extended syntax was present. 366 4.3. Using Multiple Instances for Internationalization 368 It is expected that in many cases, internationalization of parameters 369 in response headers is implemented using server driven content 370 negotiation ([RFC2616], Section 12.1) using the Accept-Language 371 header ([RFC2616], Section 14.4). However, the format described in 372 this specification also allows to use multiple instances providing 373 multiple languages in a single header. Specifications that want to 374 take advantage of this should clearly specify the expected processing 375 by the recipient. 377 Example: 379 foo: bar; title*=utf-8'en'Document%20Title; 380 title*=utf-8'de'Titel%20des%20Dokuments 382 5. Security Considerations 384 This document does not discuss security issues and is not believed to 385 raise any security issues not already endemic in HTTP. 387 6. IANA Considerations 389 There are no IANA Considerations related to this specification. 391 7. Acknowledgements 393 Thanks to Martin Duerst and Frank Ellermann for help figuring out 394 ABNF details, and to Roar Lauritzsen for implementer's feedback. 396 8. References 398 8.1. Normative References 400 [ISO-8859-1] 401 International Organization for Standardization, 402 "Information technology -- 8-bit single-byte coded graphic 403 character sets -- Part 1: Latin alphabet No. 1", ISO/ 404 IEC 8859-1:1998, 1998. 406 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 407 Requirement Levels", BCP 14, RFC 2119, March 1997. 409 [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., 410 Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext 411 Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. 413 [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 414 10646", RFC 3629, STD 63, November 2003. 416 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 417 Specifications: ABNF", STD 68, RFC 5234, January 2008. 419 [RFC5646] Phillips, A., Ed. and M. Davis, Ed., "Tags for Identifying 420 Languages", BCP 47, RFC 5646, September 2009. 422 8.2. Informative References 424 [RFC2047] Moore, K., "MIME (Multipurpose Internet Mail Extensions) 425 Part Three: Message Header Extensions for Non-ASCII Text", 426 RFC 2047, November 1996. 428 [RFC2231] Freed, N. and K. Moore, "MIME Parameter Value and Encoded 429 Word Extensions: 430 Character Sets, Languages, and Continuations", RFC 2231, 431 November 1997. 433 [RFC2277] Alvestrand, H., "IETF Policy on Character Sets and 434 Languages", BCP 18, RFC 2277, January 1998. 436 [RFC2978] Freed, N. and J. Postel, "IANA Charset Registration 437 Procedures", BCP 19, RFC 2978, October 2000. 439 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 440 Resource Identifier (URI): Generic Syntax", RFC 3986, 441 STD 66, January 2005. 443 [USASCII] American National Standards Institute, "Coded Character 444 Set -- 7-bit American Standard Code for Information 445 Interchange", ANSI X3.4, 1986. 447 URIs 449 [1] 451 [2] 453 [3] 455 Appendix A. Document History and Future Plans (to be removed by RFC 456 Editor before publication) 458 Problems with the internationalization of the HTTP Content- 459 Disposition header field have been known for many years (see test 460 cases at ). 462 During IETF 72 463 (), the 464 HTTPbis Working Group shortly discussed how to deal with the 465 underspecification of (1) Content-Disposition, and its (2) 466 internationalization aspects. Back then, there was rough consensus 467 in the room to move the definition into a separate draft. 469 This specification addresses problem (2), by defining a simple subset 470 of the encoding format defined in RFC 2231. A separate 471 specification, draft-reschke-rfc2183-in-http, is planned to address 472 problem (1). Note that this approach was chosen because Content- 473 Disposition is just an example for an HTTP header field using this 474 kind of encoding. Another example is the currently proposed Link 475 header field (draft-nottingham-http-link-header). 477 This document is planned to be published on the IETF Standards Track, 478 so that other standards-track level documents can depend on it, such 479 as the new specification of Content-Disposition, or potentially 480 future revisions of the HTTP Link Header specification. 482 Also note that this document specifies a proper subset of the 483 extensions defined in RFC 2231, but does not normatively refer to it. 484 Thus, RFC 2231 can be revised separately, should the email community 485 decide to. 487 Appendix B. Change Log (to be removed by RFC Editor before publication) 489 B.1. Since draft-reschke-rfc2231-in-http-00 491 Use RFC5234-style ABNF, closer to the one used in RFC 2231. 493 Make RFC 2231 dependency informative, so this specification can 494 evolve independently. 496 Explain the ABNF in prose. 498 B.2. Since draft-reschke-rfc2231-in-http-01 500 Remove unneeded RFC5137 notation (code point vs character). 502 B.3. Since draft-reschke-rfc2231-in-http-02 504 And and resolve issues "charset", "repeats" and "rfc4646". 506 B.4. Since draft-reschke-rfc2231-in-http-03 508 And and resolve issue "charsetmatch". 510 B.5. Since draft-reschke-rfc2231-in-http-04 512 Add and resolve issues "badseq" and "tokenquotcharset". 514 B.6. Since draft-reschke-rfc2231-in-http-05 516 Say "header field" instead of "header" in the context of HTTP. 518 B.7. Since draft-reschke-rfc2231-in-http-06 520 Add an appendix discussing document history and future plans, to be 521 removed before publication. 523 Appendix C. Open issues (to be removed by RFC Editor prior to 524 publication) 526 C.1. edit 528 Type: edit 530 julian.reschke@greenbytes.de (2009-04-17): Umbrella issue for 531 editorial fixes/enhancements. 533 Author's Address 535 Julian F. Reschke 536 greenbytes GmbH 537 Hafenweg 16 538 Muenster, NW 48155 539 Germany 541 Email: julian.reschke@greenbytes.de 542 URI: http://greenbytes.de/tech/webdav/