idnits 2.17.1 draft-ietf-websec-mime-sniff-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Sep 2009 rather than the newer Notice from 28 Dec 2009. (See https://trustee.ietf.org/license-info/) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 112: '...ble, user agents SHOULD NOT employ a c...' RFC 2119 keyword, line 114: '..., the user agent SHOULD use the algori...' RFC 2119 keyword, line 124: '...ed as algorithms or specific steps MAY...' RFC 2119 keyword, line 144: '..., the user agent MUST use the textuall...' RFC 2119 keyword, line 172: '... The user agent MUST use the followin...' (8 more instances...) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 24, 2011) is 4812 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC2046' is mentioned on line 168, but not defined -- Looks like a reference, but probably isn't: '0' on line 574 -- Looks like a reference, but probably isn't: '1' on line 574 -- Looks like a reference, but probably isn't: '2' on line 574 -- Possible downref: Non-RFC (?) normative reference: ref. 'BarthCaballeroSong2009' Summary: 5 errors (**), 0 flaws (~~), 3 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 None A. Barth 3 Internet-Draft I. Hickson 4 Expires: July 28, 2011 Google, Inc. 5 January 24, 2011 7 Media Type Sniffing 8 draft-ietf-websec-mime-sniff-01 10 Abstract 12 Many web servers supply incorrect Content-Type header fields with 13 their HTTP responses. In order to be compatible with these servers, 14 user agents consider the content of HTTP responses as well as the 15 Content-Type header fields when determining the effective media type 16 of the response. This document describes an algorithm for 17 determining the effective media type of HTTP responses that balances 18 security and compatibility considerations. 20 Please send feedback on this draft to apps-discuss@ietf.org. 22 Status of this Memo 24 This Internet-Draft is submitted to IETF in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF), its areas, and its working groups. Note that 29 other groups may also distribute working documents as Internet- 30 Drafts. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 The list of current Internet-Drafts can be accessed at 38 http://www.ietf.org/ietf/1id-abstracts.txt. 40 The list of Internet-Draft Shadow Directories can be accessed at 41 http://www.ietf.org/shadow.html. 43 This Internet-Draft will expire on July 28, 2011. 45 Copyright Notice 47 Copyright (c) 2011 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (http://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 63 2. Metadata . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 64 3. Web Pages . . . . . . . . . . . . . . . . . . . . . . . . . . 6 65 4. Text or Binary . . . . . . . . . . . . . . . . . . . . . . . . 8 66 5. Unknown Type . . . . . . . . . . . . . . . . . . . . . . . . . 10 67 6. Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 68 7. Feed or HTML . . . . . . . . . . . . . . . . . . . . . . . . . 17 69 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 20 70 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 21 72 1. Introduction 74 The HTTP Content-Type header field indicates the media type of an 75 HTTP response. However, many HTTP servers supply a Content-Type that 76 does not match the actual contents of the response. Historically, 77 web browsers have tolerated these servers by examining the content of 78 HTTP responses in addition to the Content-Type header field to 79 determine the effective media type of the response. 81 Without a clear specification of how to "sniff" the media type, each 82 user agent implementor was forced to reverse engineer the behavior of 83 the other user agents and to develop their own algorithm. These 84 divergent algorithms have lead to a lack of interoperability between 85 user agents and to security issues when the server intends an HTTP 86 response to be interpreted as one media type but some user agents 87 interpret the responses as another media type. 89 These security issues are most severe when an "honest" server lets 90 potentially malicious users upload files and then serves the contents 91 of those files with a low-privilege media type (such as text/plain or 92 image/jpeg). (Malicious servers, of course, can specify an arbitrary 93 media type in the Content-Type header field.) In the absence of 94 media type sniffing, this user-generated content would not be 95 interpreted as a high-privilege media type, such as text/html. 96 However, if a user agent does interpret a low-privilege media type, 97 such as image/gif, as a high-privilege media type, such as text/html, 98 the user agent has created a privilege escalation vulnerability in 99 the server. For example, a malicious user might be able to leverage 100 content sniffing to mount a cross-site script attack by including 101 JavaScript code in the uploaded file that a user agent treats as 102 text/html. 104 This document describes a content sniffing algorithm that carefully 105 balances the compatibility needs of user agent implementors with the 106 security constraints. The algorithm has been constructed with 107 reference to content sniffing algorithms present in popular user 108 agents, an extensive database of existing web content, and metrics 109 collected from implementations deployed to a sizable number of users 110 [BarthCaballeroSong2009]. 112 WARNING! Whenever possible, user agents SHOULD NOT employ a content 113 sniffing algorithm. However, if a user agent does employ a content 114 sniffing algorithm, the user agent SHOULD use the algorithm in this 115 document because using a different content sniffing algorithm than 116 servers expect causes security problems. For example, if a server 117 believes that the client will treat a contributed file as an image 118 (and thus treat it as benign), but a user agent believes the content 119 to be HTML (and thus privileged to execute any scripts contained 120 therein), an attacker might be able to steal the user's 121 authentication credentials and mount other cross-site scripting 122 attacks. 124 Conformance requirements phrased as algorithms or specific steps MAY 125 be implemented in any manner, so long as the end result is 126 equivalent. (In particular, the algorithms defined in this 127 specification are intended to be easy to follow, and not intended to 128 be performant.) 130 2. Metadata 132 The explicit media type metadata information associated with sequence 133 of octets depends on the protocol that was used to fetch the octets. 135 For octets received via HTTP, the Content-Type HTTP header field, if 136 present, indicates the media type. Let the official-type be the 137 media type indicted by the HTTP Content-Type header field, if 138 present. If the Content-Type header field is absent or if its value 139 cannot be interpreted as a media type (e.g. because its value doesn't 140 contain a U+002F SOLIDUS ('/') character), then there is no official- 141 type. 143 Note: If an HTTP response contains multiple Content-Type header 144 fields, the user agent MUST use the textually last Content-Type 145 header field to the official-type. For example, if the last 146 Content-Type header field contains the value "foo", then there is 147 no official media type because "foo" cannot be interpreted as a 148 media type (even if the HTTP response contains another Content- 149 Type header field that could be interpreted as a media type). 151 For octets fetched from the file system, user agents should use 152 platform-specific conventions (e.g., operating system file extension/ 153 type mappings) to determine the official-type. 155 Note: It is essential that file extensions are not used for 156 determining the media type for octets fetched over HTTP because, 157 in some cases, file extensions can be supplied by malicious 158 parties. For example, most PHP installations let the attacker 159 append arbitrary path information to URLs (e.g., 160 http://example.com/foo.php/bar.html) and thereby determine the 161 file extension. 163 For octets fetched over some other protocols, e.g. FTP, there is no 164 type information. 166 Note: Comparisons between media types, as defined by MIME 167 specifications, are done in an ASCII case-insensitive manner. 168 [RFC2046] 170 3. Web Pages 172 The user agent MUST use the following algorithm to determine the 173 sniffed-type of a sequence of octets: 175 1. If the user agent is configured to strictly obey the official- 176 type, then let the sniffed-type be the official-type and abort 177 these steps. 179 2. If the octets were fetched via HTTP and there is an HTTP Content- 180 Type header field and the value of the last such header field has 181 octets that *exactly* match the octets contained in one of the 182 following lines: 184 +-------------------------------+--------------------------------+ 185 | Bytes in Hexadecimal | Textual Representation | 186 +-------------------------------+--------------------------------+ 187 | 74 65 78 74 2f 70 6c 61 69 6e | text/plain | 188 +-------------------------------+--------------------------------+ 189 | 74 65 78 74 2f 70 6c 61 69 6e | text/plain; charset=ISO-8859-1 | 190 | 3b 20 63 68 61 72 73 65 74 3d | | 191 | 49 53 4f 2d 38 38 35 39 2d 31 | | 192 +-------------------------------+--------------------------------+ 193 | 74 65 78 74 2f 70 6c 61 69 6e | text/plain; charset=iso-8859-1 | 194 | 3b 20 63 68 61 72 73 65 74 3d | | 195 | 69 73 6f 2d 38 38 35 39 2d 31 | | 196 +-------------------------------+--------------------------------+ 197 | 74 65 78 74 2f 70 6c 61 69 6e | text/plain; charset=UTF-8 | 198 | 3b 20 63 68 61 72 73 65 74 3d | | 199 | 55 54 46 2d 38 | | 200 +-------------------------------+--------------------------------+ 202 ...then jump to the "text or binary" section below. 204 3. If there is no official-type, jump to the "unknown type" section 205 below. 207 4. If the official-type is "unknown/unknown", "application/unknown", 208 or "*/*", jump to the "unknown" type section below. 210 5. If the official-type ends in "+xml", or if it is either "text/ 211 xml" or "application/xml", then let the sniffed-type be the 212 official-type and abort these steps. 214 6. If the official-type is an image type supported by the user agent 215 (e.g., "image/png", "image/gif", "image/jpeg", etc), then jump to 216 the "images" section below. 218 7. If the official-type is "text/html", then jump to the "feed or 219 HTML" section below. 221 8. Let the sniffed-type be the official type. 223 4. Text or Binary 225 This section defines the *rules for distingushing if a resource is 226 text or binary*. 228 1. The user agent MAY wait for 512 or more octets be to arrive. 230 Note: Waiting for 512 octets octets to arrive causes the text- 231 or-binary algorithm to be deterministic for a given sequence 232 of octets. However, in some cases, the user agent might need 233 to wait an arbitrary length of time for these octets to 234 arrive. User agents SHOULD wait for 512 octets to arrive, 235 when feasible. 237 2. Let n be the smaller of either 512 or the number of octets that 238 have already arrived. 240 3. If n is greater than or equal to 3, and the first 2 or 3 octets 241 match one of the following octet sequences: 243 +----------------------+--------------+ 244 | Bytes in Hexadecimal | Description | 245 +----------------------+--------------+ 246 | FE FF | UTF-16BE BOM | 247 | FF FE | UTF-16LE BOM | 248 | EF BB BF | UTF-8 BOM | 249 +----------------------+--------------+ 251 ...then let the sniffed-type be "text/plain" and abort these 252 steps. 254 4. If none of the first n octets are binary data octets then let the 255 sniffed-type be "text/plain" and abort these steps. 257 +-------------------------+ 258 | Binary Data Byte Ranges | 259 +-------------------------+ 260 | 0x00 -- 0x08 | 261 | 0x0B | 262 | 0x0E -- 0x1A | 263 | 0x1C -- 0x1F | 264 +-------------------------+ 266 5. If the first octets match one of the octet sequences in the 267 "pattern" column of the table in the "unknown type" section 268 below, ignoring any rows whose cell in the "security" column says 269 "scriptable" (or "n/a"), then let the sniffed-type be the type 270 given in the corresponding cell in the "sniffed type" column on 271 that row and abort these steps. 273 WARNING! It is critical that this step not ever return a 274 scriptable type (e.g., text/html), because otherwise that 275 would allow a privilege escalation attack. 277 6. Otherwise, let the sniffed-type be "application/octet-stream" and 278 abort these steps. 280 5. Unknown Type 282 1. The user agent MAY wait for 512 or more octets to arrive for the 283 same reason as in the "text or binary" section above. 285 2. Let n be the smaller of either 512 or the number of octets that 286 have already arrived. 288 3. For each row in the table below: 290 * If the row has no "WS" octets: 292 1. Let pattern-length be the length of the pattern. 294 2. If n is smaller than pattern-length then skip this row. 296 3. Apply the bit-wise "and" operator to the first pattern- 297 length octets and the given mask, and let the result be 298 the masked-data. 300 4. If the octets of the masked-data matches the given pattern 301 octets exactly, then let the sniffed-type be the type 302 given in the cell of the third column in that row and 303 abort these steps. 305 * If the row has a "WS" octet or a "_>" octet: 307 1. Let index-pattern be an index into the mask and pattern 308 octet strings of the row. 310 2. Let index-stream be an index into the octet stream being 311 examined. 313 3. LOOP: If index-stream points beyond the end of the octet 314 stream, then this row doesn't match and skip this row. 316 4. Examine the index-stream-th octet of the octet stream as 317 follows: 319 - If the index-pattern-th octet of the pattern is a 320 normal hexadecimal octet and not a "WS" octet or a "SB" 321 octet: 323 If the bit-wise "and" operator, applied to the 324 index-stream-th octet of the stream and the index- 325 pattern-th octet of the mask, yield a value 326 different than the index-pattern-th octet of the 327 pattern, then skip this row. 329 Otherwise, increment index-pattern to the next octet 330 in the mask and pattern and index-stream to the next 331 octet in the octet stream. 333 - Otherwise, if the index-pattern-th octet of the pattern 334 is a "WS" octet: 336 "WS" means "whitespace", and allows insignificant 337 whitespace to be skipped when sniffing for a type 338 signature. 340 If the index-stream-th octet of the stream is one of 341 0x09 (ASCII TAB), 0x0A (ASCII LF), 0x0C (ASCII FF), 342 0x0D (ASCII CR), or 0x20 (ASCII space), then 343 increment only the index-stream to the next octet in 344 the octet stream. 346 Otherwise, increment only the index-pattern to the 347 next octet in the mask and pattern. 349 - Otherwise, if the index-pattern-th octet of the pattern 350 is a "_>" octet: 352 "_>" means "space-or-bracket", and allows HTML tag 353 names to terminate with either a space or a greater 354 than sign. 356 If index-stream-th octet of the stream different 357 than 0x20 (ASCII space) or 0x3E (ASCII ">"), then 358 skip this row. 360 Otherwise, increment index-pattern to the next octet 361 in the mask and pattern and index-stream to the next 362 octet in the octet stream. 364 5. If index-pattern does not point beyond the end of the mask 365 and pattern octet strings, then jump back to the LOOP step 366 in this algorithm. 368 6. Otherwise, let the sniffed-type be the type given in the 369 cell of the third column in that row and abort these 370 steps. 372 4. If none of the first n octets are binary data (as defined in the 373 "text or binary" section), then let the sniffed-type be "text/ 374 plain" and abort these steps. 376 5. Otherwise, let the sniffed-type be "application/octet-stream" and 377 abort these steps. 379 The table used by the above algorithm is: 381 +-------------------+-------------------+-----------------+------------+ 382 | Mask in Hex | Pattern in Hex | Sniffed Type | Security | 383 +-------------------+-------------------+-----------------+------------+ 384 | FF FF FF DF DF DF | WS 3C 21 44 4F 43 | text/html | Scriptable | 385 | DF DF DF DF FF DF | 54 59 50 45 20 48 | | | 386 | DF DF DF FF | 54 4D 4C _> | | | 387 | Comment: | | | 391 | Comment: | | | 395 | Comment: | | | 399 | Comment: