idnits 2.17.1 

draft-abarth-mime-sniff-05.txt:

  Checking boilerplate required by RFC 5378 and the IETF Trust (see
  https://trustee.ietf.org/license-info):
  ----------------------------------------------------------------------------

  ** You're using the IETF Trust Provisions' Section 6.b License Notice from
     12 Sep 2009 rather than the newer Notice from 28 Dec 2009.  (See
     https://trustee.ietf.org/license-info/)


  Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
  ----------------------------------------------------------------------------

  == No 'Intended status' indicated for this document; assuming Proposed
     Standard


  Checking nits according to https://www.ietf.org/id-info/checklist :
  ----------------------------------------------------------------------------

  ** The document seems to lack a Security Considerations section.

  ** The document seems to lack an IANA Considerations section.  (See Section
     2.2 of https://www.ietf.org/id-info/checklist for how to handle the case
     when there are no actions for IANA.)

  ** The document seems to lack separate sections for Informative/Normative
     References.  All references will be assumed normative when checking for
     downward references.

  ** The document seems to lack a both a reference to RFC 2119 and the
     recommended RFC 2119 boilerplate, even if it appears to use RFC 2119
     keywords. 

     RFC 2119 keyword, line 113: '...ble, user agents SHOULD NOT employ a c...'
     RFC 2119 keyword, line 115: '..., the user agent SHOULD use the algori...'
     RFC 2119 keyword, line 125: '...ed as algorithms or specific steps MAY...'
     RFC 2119 keyword, line 145: '..., the user agent MUST use the textuall...'
     RFC 2119 keyword, line 173: '...  The user agent MUST use the followin...'
     (8 more instances...)


  Miscellaneous warnings:
  ----------------------------------------------------------------------------

  == The copyright year in the IETF Trust and authors Copyright Line does not
     match the current year

  -- The document date (May 4, 2010) is 5099 days in the past.  Is this
     intentional?

  -- Found something which looks like a code comment -- if you have code
     sections in the document, please surround them with '<CODE BEGINS>' and
     '<CODE ENDS>' lines.


  Checking references for intended status: Proposed Standard
  ----------------------------------------------------------------------------

     (See RFCs 3967 and 4897 for information about using normative references
     to lower-maturity documents in RFCs)

  == Missing Reference: 'RFC2046' is mentioned on line 169, but not defined

  -- Looks like a reference, but probably isn't: '0' on line 558

  -- Looks like a reference, but probably isn't: '1' on line 558

  -- Looks like a reference, but probably isn't: '2' on line 558

  -- Possible downref: Non-RFC (?) normative reference: ref.
     'BarthCaballeroSong2009'


     Summary: 5 errors (**), 0 flaws (~~), 3 warnings (==), 6 comments (--).

     Run idnits with the --verbose option for more detailed information about
     the items above.

--------------------------------------------------------------------------------


2	None                                                            A. Barth
3	Internet-Draft                                             U.C. Berkeley
4	Expires: November 5, 2010                                     I. Hickson
5	                                                            Google, Inc.
6	                                                             May 4, 2010

8	                          Media Type Sniffing
9	                       draft-abarth-mime-sniff-05

11	Abstract

13	   Many web servers supply incorrect Content-Type header fields with
14	   their HTTP responses.  In order to be compatible with these servers,
15	   user agents consider the content of HTTP responses as well as the
16	   Content-Type header fields when determining the effective media type
17	   of the response.  This document describes an algorithm for
18	   determining the effective media type of HTTP responses that balances
19	   security and compatibility considerations.

21	   Please send feedback on this draft to apps-discuss@ietf.org.

23	Status of this Memo

25	   This Internet-Draft is submitted to IETF in full conformance with the
26	   provisions of BCP 78 and BCP 79.

28	   Internet-Drafts are working documents of the Internet Engineering
29	   Task Force (IETF), its areas, and its working groups.  Note that
30	   other groups may also distribute working documents as Internet-
31	   Drafts.

33	   Internet-Drafts are draft documents valid for a maximum of six months
34	   and may be updated, replaced, or obsoleted by other documents at any
35	   time.  It is inappropriate to use Internet-Drafts as reference
36	   material or to cite them other than as "work in progress."

38	   The list of current Internet-Drafts can be accessed at
39	   http://www.ietf.org/ietf/1id-abstracts.txt.

41	   The list of Internet-Draft Shadow Directories can be accessed at
42	   http://www.ietf.org/shadow.html.

44	   This Internet-Draft will expire on November 5, 2010.

46	Copyright Notice

48	   Copyright (c) 2010 IETF Trust and the persons identified as the
49	   document authors.  All rights reserved.

51	   This document is subject to BCP 78 and the IETF Trust's Legal
52	   Provisions Relating to IETF Documents
53	   (http://trustee.ietf.org/license-info) in effect on the date of
54	   publication of this document.  Please review these documents
55	   carefully, as they describe your rights and restrictions with respect
56	   to this document.  Code Components extracted from this document must
57	   include Simplified BSD License text as described in Section 4.e of
58	   the Trust Legal Provisions and are provided without warranty as
59	   described in the BSD License.

61	Table of Contents

63	   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
64	   2.  Metadata . . . . . . . . . . . . . . . . . . . . . . . . . . .  5
65	   3.  Web Pages  . . . . . . . . . . . . . . . . . . . . . . . . . .  6
66	   4.  Text or Binary . . . . . . . . . . . . . . . . . . . . . . . .  8
67	   5.  Unknown Type . . . . . . . . . . . . . . . . . . . . . . . . . 10
68	   6.  Image  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
69	   7.  Feed or HTML . . . . . . . . . . . . . . . . . . . . . . . . . 16
70	   8.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 19
71	   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 20

73	1.  Introduction

75	   The HTTP Content-Type header field indicates the media type of an
76	   HTTP response.  However, many HTTP servers supply a Content-Type that
77	   does not match the actual contents of the response.  Historically,
78	   web browsers have tolerated these servers by examining the content of
79	   HTTP responses in addition to the Content-Type header field to
80	   determine the effective media type of the response.

82	   Without a clear specification of how to "sniff" the media type, each
83	   user agent implementor was forced to reverse engineer the behavior of
84	   the other user agents and to develop their own algorithm.  These
85	   divergent algorithms have lead to a lack of interoperability between
86	   user agents and to security issues when the server intends an HTTP
87	   response to be interpreted as one media type but some user agents
88	   interpret the responses as another media type.

90	   These security issues are most severe when an "honest" server lets
91	   potentially malicious users upload files and then serves the contents
92	   of those files with a low-privilege media type (such as text/plain or
93	   image/jpeg).  (Malicious servers, of course, can specify an arbitrary
94	   media type in the Content-Type header field.)  In the absence of
95	   media type sniffing, this user-generated content would not be
96	   interpreted as a high-privilege media type, such as text/html.
97	   However, if a user agent does interpret a low-privilege media type,
98	   such as image/gif, as a high-privilege media type, such as text/html,
99	   the user agent has created a privilege escalation vulnerability in
100	   the server.  For example, a malicious user might be able to leverage
101	   content sniffing to mount a cross-site script attack by including
102	   JavaScript code in the uploaded file that a user agent treats as
103	   text/html.

105	   This document describes a content sniffing algorithm that carefully
106	   balances the compatibility needs of user agent implementors with the
107	   security constraints.  The algorithm has been constructed with
108	   reference to content sniffing algorithms present in popular user
109	   agents, an extensive database of existing web content, and metrics
110	   collected from implementations deployed to a sizable number of users
111	   [BarthCaballeroSong2009].

113	   WARNING!  Whenever possible, user agents SHOULD NOT employ a content
114	   sniffing algorithm.  However, if a user agent does employ a content
115	   sniffing algorithm, the user agent SHOULD use the algorithm in this
116	   document because using a different content sniffing algorithm than
117	   servers expect causes security problems.  For example, if a server
118	   believes that the client will treat a contributed file as an image
119	   (and thus treat it as benign), but a user agent believes the content
120	   to be HTML (and thus privileged to execute any scripts contained
121	   therein), an attacker might be able to steal the user's
122	   authentication credentials and mount other cross-site scripting
123	   attacks.

125	   Conformance requirements phrased as algorithms or specific steps MAY
126	   be implemented in any manner, so long as the end result is
127	   equivalent.  (In particular, the algorithms defined in this
128	   specification are intended to be easy to follow, and not intended to
129	   be performant.)

131	2.  Metadata

133	   The explicit media type metadata information associated with sequence
134	   of octets depends on the protocol that was used to fetch the octets.

136	   For octets received via HTTP, the Content-Type HTTP header field, if
137	   present, indicates the media type.  Let the official-type be the
138	   media type indicted by the HTTP Content-Type header field, if
139	   present.  If the Content-Type header field is absent or if its value
140	   cannot be interpreted as a media type (e.g. because its value doesn't
141	   contain a U+002F SOLIDUS ('/') character), then there is no official-
142	   type.

144	      Note: If an HTTP response contains multiple Content-Type header
145	      fields, the user agent MUST use the textually last Content-Type
146	      header field to the official-type.  For example, if the last
147	      Content-Type header field contains the value "foo", then there is
148	      no official media type because "foo" cannot be interpreted as a
149	      media type (even if the HTTP response contains another Content-
150	      Type header field that could be interpreted as a media type).

152	   For octets fetched from the file system, user agents should use
153	   platform-specific conventions (e.g., operating system file extension/
154	   type mappings) to determine the official-type.

156	      Note: It is essential that file extensions are not used for
157	      determining the media type for octets fetched over HTTP because,
158	      in some cases, file extensions can be supplied by malicious
159	      parties.  For example, most PHP installations let the attacker
160	      append arbitrary path information to URLs (e.g.,
161	      http://example.com/foo.php/bar.html) and thereby determine the
162	      file extension.

164	   For octets fetched over some other protocols, e.g.  FTP, there is no
165	   type information.

167	   Note: Comparisons between media types, as defined by MIME
168	   specifications, are done in an ASCII case-insensitive manner.
169	   [RFC2046]

171	3.  Web Pages

173	   The user agent MUST use the following algorithm to determine the
174	   sniffed-type of a sequence of octets:

176	   1.  If the user agent is configured to strictly obey the official-
177	       type, then let the sniffed-type be the official-type and abort
178	       these steps.

180	   2.  If the octets were fetched via HTTP and there is an HTTP Content-
181	       Type header field and the value of the last such header field has
182	       octets that *exactly* match the octets contained in one of the
183	       following lines:

185	      +-------------------------------+--------------------------------+
186	      | Bytes in Hexadecimal          | Textual Representation         |
187	      +-------------------------------+--------------------------------+
188	      | 74 65 78 74 2f 70 6c 61 69 6e | text/plain                     |
189	      +-------------------------------+--------------------------------+
190	      | 74 65 78 74 2f 70 6c 61 69 6e | text/plain; charset=ISO-8859-1 |
191	      | 3b 20 63 68 61 72 73 65 74 3d |                                |
192	      | 49 53 4f 2d 38 38 35 39 2d 31 |                                |
193	      +-------------------------------+--------------------------------+
194	      | 74 65 78 74 2f 70 6c 61 69 6e | text/plain; charset=iso-8859-1 |
195	      | 3b 20 63 68 61 72 73 65 74 3d |                                |
196	      | 69 73 6f 2d 38 38 35 39 2d 31 |                                |
197	      +-------------------------------+--------------------------------+
198	      | 74 65 78 74 2f 70 6c 61 69 6e | text/plain; charset=UTF-8      |
199	      | 3b 20 63 68 61 72 73 65 74 3d |                                |
200	      | 55 54 46 2d 38                |                                |
201	      +-------------------------------+--------------------------------+

203	       ...then jump to the "text or binary" section below.

205	   3.  If there is no official-type, jump to the "unknown type" section
206	       below.

208	   4.  If the official-type is "unknown/unknown", "application/unknown",
209	       or "*/*", jump to the "unknown" type section below.

211	   5.  If the official-type ends in "+xml", or if it is either "text/
212	       xml" or "application/xml", then let the sniffed-type be the
213	       official-type and abort these steps.

215	   6.  If the official-type is an image type supported by the user agent
216	       (e.g., "image/png", "image/gif", "image/jpeg", etc), then jump to
217	       the "images" section below.

219	   7.  If the official-type is "text/html", then jump to the "feed or
220	       HTML" section below.

222	   8.  Let the sniffed-type be the official type.

224	4.  Text or Binary

226	   This section defines the *rules for distingushing if a resource is
227	   text or binary*.

229	   1.  The user agent MAY wait for 512 or more octets be to arrive.

231	          Note: Waiting for 512 octets octets to arrive causes the text-
232	          or-binary algorithm to be deterministic for a given sequence
233	          of octets.  However, in some cases, the user agent might need
234	          to wait an arbitrary length of time for these octets to
235	          arrive.  User agents SHOULD wait for 512 octets to arrive,
236	          when feasible.

238	   2.  Let n be the smaller of either 512 or the number of octets that
239	       have already arrived.

241	   3.  If n is greater than or equal to 3, and the first 2 or 3 octets
242	       match one of the following octet sequences:

244	                   +----------------------+--------------+
245	                   | Bytes in Hexadecimal | Description  |
246	                   +----------------------+--------------+
247	                   | FE FF                | UTF-16BE BOM |
248	                   | FF FE                | UTF-16LE BOM |
249	                   | EF BB BF             | UTF-8 BOM    |
250	                   +----------------------+--------------+

252	       ...then let the sniffed-type be "text/plain" and abort these
253	       steps.

255	   4.  If none of the first n octets are binary data octets then let the
256	       sniffed-type be "text/plain" and abort these steps.

258	                         +-------------------------+
259	                         | Binary Data Byte Ranges |
260	                         +-------------------------+
261	                         | 0x00 -- 0x08            |
262	                         | 0x0B                    |
263	                         | 0x0E -- 0x1A            |
264	                         | 0x1C -- 0x1F            |
265	                         +-------------------------+

267	   5.  If the first octets match one of the octet sequences in the
268	       "pattern" column of the table in the "unknown type" section
269	       below, ignoring any rows whose cell in the "security" column says
270	       "scriptable" (or "n/a"), then let the sniffed-type be the type
271	       given in the corresponding cell in the "sniffed type" column on
272	       that row and abort these steps.

274	          WARNING!  It is critical that this step not ever return a
275	          scriptable type (e.g., text/html), because otherwise that
276	          would allow a privilege escalation attack.

278	   6.  Otherwise, let the sniffed-type be "application/octet-stream" and
279	       abort these steps.

281	5.  Unknown Type

283	   1.  The user agent MAY wait for 512 or more octets to arrive for the
284	       same reason as in the "text or binary" section above.

286	   2.  Let n be the smaller of either 512 or the number of octets that
287	       have already arrived.

289	   3.  For each row in the table below:

291	       *  If the row has no "WS" octets:

293	          1.  Let pattern-length be the length of the pattern.

295	          2.  If n is smaller than pattern-length then skip this row.

297	          3.  Apply the bit-wise "and" operator to the first pattern-
298	              length octets and the given mask, and let the result be
299	              the masked-data.

301	          4.  If the octets of the masked-data matches the given pattern
302	              octets exactly, then let the sniffed-type be the type
303	              given in the cell of the third column in that row and
304	              abort these steps.

306	       *  If the row has a "WS" octet or a "_>" octet:

308	          1.  Let index-pattern be an index into the mask and pattern
309	              octet strings of the row.

311	          2.  Let index-stream be an index into the octet stream being
312	              examined.

314	          3.  LOOP: If index-stream points beyond the end of the octet
315	              stream, then this row doesn't match and skip this row.

317	          4.  Examine the index-stream-th octet of the octet stream as
318	              follows:

320	              -  If the index-pattern-th octet of the pattern is a
321	                 normal hexadecimal octet and not a "WS" octet or a "SB"
322	                 octet:

324	                    If the bit-wise "and" operator, applied to the
325	                    index-stream-th octet of the stream and the index-
326	                    pattern-th octet of the mask, yield a value
327	                    different than the index-pattern-th octet of the
328	                    pattern, then skip this row.

330	                    Otherwise, increment index-pattern to the next octet
331	                    in the mask and pattern and index-stream to the next
332	                    octet in the octet stream.

334	              -  Otherwise, if the index-pattern-th octet of the pattern
335	                 is a "WS" octet:

337	                    "WS" means "whitespace", and allows insignificant
338	                    whitespace to be skipped when sniffing for a type
339	                    signature.

341	                    If the index-stream-th octet of the stream is one of
342	                    0x09 (ASCII TAB), 0x0A (ASCII LF), 0x0C (ASCII FF),
343	                    0x0D (ASCII CR), or 0x20 (ASCII space), then
344	                    increment only the index-stream to the next octet in
345	                    the octet stream.

347	                    Otherwise, increment only the index-pattern to the
348	                    next octet in the mask and pattern.

350	              -  Otherwise, if the index-pattern-th octet of the pattern
351	                 is a "_>" octet:

353	                    "_>" means "space-or-bracket", and allows HTML tag
354	                    names to terminate with either a space or a greater
355	                    than sign.

357	                    If index-stream-th octet of the stream different
358	                    than 0x20 (ASCII space) or 0x3E (ASCII ">"), then
359	                    skip this row.

361	                    Otherwise, increment index-pattern to the next octet
362	                    in the mask and pattern and index-stream to the next
363	                    octet in the octet stream.

365	          5.  If index-pattern does not point beyond the end of the mask
366	              and pattern octet strings, then jump back to the LOOP step
367	              in this algorithm.

369	          6.  Otherwise, let the sniffed-type be the type given in the
370	              cell of the third column in that row and abort these
371	              steps.

373	   4.  If none of the first n octets are binary data (as defined in the
374	       "text or binary" section), then let the sniffed-type be "text/
375	       plain" and abort these steps.

377	   5.  Otherwise, let the sniffed-type be "application/octet-stream" and
378	       abort these steps.

380	   The table used by the above algorithm is:

382	+-------------------+-------------------+-----------------+------------+
383	| Mask in Hex       | Pattern in Hex    | Sniffed Type    | Security   |
384	+-------------------+-------------------+-----------------+------------+
385	| FF FF FF DF DF DF | WS 3C 21 44 4F 43 | text/html       | Scriptable |
386	| DF DF DF DF FF DF | 54 59 50 45 20 48 |                 |            |
387	| DF DF DF FF       | 54 4D 4C _>       |                 |            |
388	| Comment: <!DOCTYPE HTML                                              |
389	+-------------------+-------------------+-----------------+------------+
390	| FF FF DF DF DF DF | WS 3C 48 54 4D 4C | text/html       | Scriptable |
391	| FF                | _>                |                 |            |
392	| Comment: <HTML                                                       |
393	+-------------------+-------------------+-----------------+------------+
394	| FF FF DF DF DF DF | WS 3C 48 45 41 44 | text/html       | Scriptable |
395	| FF                | _>                |                 |            |
396	| Comment: <HEAD                                                       |
397	+-------------------+-------------------+-----------------+------------+
398	| FF FF DF DF DF DF | WS 3C 53 43 52 49 | text/html       | Scriptable |
399	| DF DF FF          | 50 54 _>          |                 |            |
400	| Comment: <SCRIPT                                                     |
401	+-------------------+-------------------+-----------------+------------+
402	| FF FF DF DF DF DF | WS 3C 49 46 52 41 | text/html       | Scriptable |
403	| DF DF FF          | 4d 45 _>          |                 |            |
404	| Comment: <IFRAME                                                     |
405	+-------------------+-------------------+-----------------+------------+
406	| FF FF DF FF FF    | WS 3C 48 31 _>    | text/html       | Scriptable |
407	| Comment: <H1                                                         |
408	+-------------------+-------------------+-----------------+------------+
409	| FF FF DF DF DF FF | WS 3C 44 49 56 _> | text/html       | Scriptable |
410	| Comment: <DIV                                                        |
411	+-------------------+-------------------+-----------------+------------+
412	| FF FF DF DF DF DF | WS 3C 46 4f 4e 54 | text/html       | Scriptable |
413	| FF                | _>                |                 |            |
414	| Comment: <FONT                                                       |
415	+-------------------+-------------------+-----------------+------------+
416	| FF FF DF DF DF DF | WS 3C 54 41 42 4c | text/html       | Scriptable |
417	| DF FF             | 45 _>             |                 |            |
418	| Comment: <TABLE                                                      |
419	+-------------------+-------------------+-----------------+------------+
420	| FF FF DF FF       | WS 3C 41 _>       | text/html       | Scriptable |
421	| Comment: <A                                                          |
422	+-------------------+-------------------+-----------------+------------+
423	| FF FF DF DF DF DF | WS 3C 53 54 59 4c | text/html       | Scriptable |
424	| DF FF             | 45 _>             |                 |            |
425	| Comment: <STYLE                                                      |
426	+-------------------+-------------------+-----------------+------------+
427	| FF FF DF DF DF DF | WS 3C 54 49 54 4c | text/html       | Scriptable |
428	| DF FF             | 45 _>             |                 |            |
429	| Comment: <TITLE                                                      |
430	+-------------------+-------------------+-----------------+------------+
431	| FF FF DF FF       | WS 3C 42 _>       | text/html       | Scriptable |
432	| Comment: <B                                                          |
433	+-------------------+-------------------+-----------------+------------+
434	| FF FF DF DF DF DF | WS 3C 42 4f 44 59 | text/html       | Scriptable |
435	| FF                | _>                |                 |            |
436	| Comment: <BODY                                                       |
437	+-------------------+-------------------+-----------------+------------+
438	| FF FF DF DF FF    | WS 3C 42 52 _>    | text/html       | Scriptable |
439	| Comment: <BR                                                         |
440	+-------------------+-------------------+-----------------+------------+
441	| FF FF DF FF       | WS 3C 50 _>       | text/html       | Scriptable |
442	| Comment: <P                                                          |
443	+-------------------+-------------------+-----------------+------------+
444	| FF FF FF FF FF FF | WS 3C 21 2d 2d _> | text/html       | Scriptable |
445	| Comment: <!--                                                        |
446	+-------------------+-------------------+-----------------+------------+
447	| FF FF FF FF FF FF | WS 3C 3f 78 6d 6c | text/xml        | Scriptable |
448	| Comment: <?xml (Note the case sensitivity and lack of trailing _>)  |
449	+-------------------+-------------------+-----------------+------------+
450	| FF FF FF FF FF    | 25 50 44 46 2D    | application/pdf | Scriptable |
451	| Comment: The string "%PDF-", the PDF signature.                      |
452	+-------------------+-------------------+-----------------+------------+
453	| FF FF FF FF FF FF | 25 21 50 53 2D 41 | application/    | Safe       |
454	| FF FF FF FF FF    | 64 6F 62 65 2D    |      postscript |            |
455	| Comment: The string "%!PS-Adobe-", the PostScript signature.         |
456	+-------------------+-------------------+-----------------+------------+
457	| FF FF 00 00       | FE FF 00 00       | text/plain      | n/a        |
458	| Comment: UTF-16BE BOM                                                |
459	+-------------------+-------------------+-----------------+------------+
460	| FF FF 00 00       | FF FE 00 00       | text/plain      | n/a        |
461	| Comment: UTF-16LE BOM                                                |
462	+-------------------+-------------------+-----------------+------------+
463	| FF FF FF 00       | EF BB BF 00       | text/plain      | n/a        |
464	| Comment: UTF-8 BOM                                                   |
465	+-------------------+-------------------+-----------------+------------+
466	| FF FF FF FF FF FF | 47 49 46 38 37 61 | image/gif       | Safe       |
467	| Comment: The string "GIF87a", a GIF signature.                       |
468	+-------------------+-------------------+-----------------+------------+
469	| FF FF FF FF FF FF | 47 49 46 38 39 61 | image/gif       | Safe       |
470	| Comment: The string "GIF89a", a GIF signature.                       |
471	+-------------------+-------------------+-----------------+------------+
472	| FF FF FF FF FF FF | 89 50 4E 47 0D 0A | image/png       | Safe       |
473	| FF FF             | 1A 0A             |                 |            |
474	| Comment: The PNG signature.                                          |
475	+-------------------+-------------------+-----------------+------------+
476	| FF FF FF          | FF D8 FF          | image/jpeg      | Safe       |
477	| Comment: A JPEG SOI marker followed by a octet of another marker.    |
478	+-------------------+-------------------+-----------------+------------+
479	| FF FF             | 42 4D             | image/bmp       | Safe       |
480	| Comment: The string "BM", a BMP signature.                           |
481	+-------------------+-------------------+-----------------+------------+
482	| FF FF FF FF       | 00 00 01 00       | image/vnd.      | Safe       |
483	|                   |                   |  microsoft.icon |            |
484	| Comment: A Windows Icon signature.                                   |
485	+-------------------+-------------------+-----------------+------------+
486	| FF FF FF FF FF FF | 52 61 72 20 1A 07 | application/    | Safe       |
487	| FF                | 00                | x-rar-compressed|            |
488	| Comment: A RAR archive.                                              |
489	+-------------------+-------------------+-----------------+------------+
490	| FF FF FF FF       | 50 4B 03 04       | application/zip | Safe       |
491	| Comment: A ZIP archive.                                              |
492	+-------------------+-------------------+-----------------+------------+
493	| FF FF FF          | 1F 8B 08          | application/    | Safe       |
494	|                   |                   |          x-gzip |            |
495	| Comment: A GZIP archive.                                             |
496	+-------------------+-------------------+-----------------+------------+

498	   User agents MAY support additional types if necessary, by implicitly
499	   adding to the above table.  However, user agents SHOULD NOT not use
500	   any other patterns for types already mentioned in the table above
501	   because this could then be used for privilege escalation (where,
502	   e.g., a server uses the above table to determine that content is not
503	   HTML and thus safe from cross-site scripting attacks, but then a user
504	   agent detects it as HTML anyway and allows script to execute).  In
505	   extending this table, user agents SHOULD NOT introduce any privilege
506	   escalation vulnerabilities.

508	   Note: The column marked "security" is used by the algorithm in the
509	   "text or binary" section, to avoid sniffing text/plain content as a
510	   type that can be used for a privilege escalation attack.

512	6.  Image

514	   This section defines the *rules for sniffing images specifically*.

516	   If the official-type is "image/svg+xml", then let the sniffed-type be
517	   the official-type (an XML type) and abort these steps.

519	   If the first octets match one of the octet sequences in the first
520	   column of the following table, then let the sniffed-type be the type
521	   given in the corresponding cell in the second column on the same row
522	   and abort these steps:

524	     +-------------------------+--------------------------+----------+
525	     | Bytes in Hexadecimal    | Sniffed Type             | Comment  |
526	     +-------------------------+--------------------------+----------+
527	     | 47 49 46 38 37 61       | image/gif                | "GIF87a" |
528	     | 47 49 46 38 39 61       | image/gif                | "GIF89a" |
529	     | 89 50 4E 47 0D 0A 1A 0A | image/png                |          |
530	     | FF D8 FF                | image/jpeg               |          |
531	     | 42 4D                   | image/bmp                | "BM"     |
532	     | 00 00 01 00             | image/vnd.microsoft.icon |          |
533	     +-------------------------+--------------------------+----------+

535	   Otherwise, let the sniffed-type be the official-type and abort these
536	   steps.

538	7.  Feed or HTML

540	   1.   The user agent MAY wait for 512 or more octets to arrive for the
541	        same reason as in the "text or binary" section above.

543	   2.   Let s be the stream of octets, and let s[i] represent the octet
544	        in s with position i, treating s as zero-indexed (so the first
545	        octet is at i=0).

547	   3.   If at any point this algorithm requires the user agent to
548	        determine the value of a octet in s which has not yet arrived,
549	        or which is past the first 512 octets, or which is beyond the
550	        end of the octet stream, the algorithm stops and the sniffed-
551	        type is "text/html".

553	           Note: User agents are allowed, by the first step of this
554	           algorithm, to wait until the first 512 octets have arrived.

556	   4.   Initialize pos to 0.

558	   5.   If s[0] equals 0xEF, s[1] equals 0xBB, and s[2] equals 0xBF,
559	        then set pos to 3.  (This skips over a leading UTF-8 BOM, if
560	        any.)

562	   6.   LOOP: Examine s[pos].

564	        *  If it equals 0x09 (ASCII tab), 0x20 (ASCII space), 0x0A
565	           (ASCII LF), or 0x0D (ASCII CR)

567	              Increase pos by 1 and repeat this step.

569	        *  If it equals 0x3C (ASCII "<")

571	              Increase pos by 1 and go to the next step.

573	        *  If it is anything else

575	              Let the sniffed-type be "text/html" and abort these steps.

577	   7.   If the octets with positions pos to pos+2 in s are exactly equal
578	        to 0x21, 0x2D, 0x2D respectively (ASCII for "!--"), then:

580	        1.  Increase pos by 3.

582	        2.  If the octets with positions pos to pos+2 in s are exactly
583	            equal to 0x2D, 0x2D, 0x3E respectively (ASCII for "-->"),
584	            then increase pos by 3 and jump back to the previous step
585	            (the step labeled loop start) in the overall algorithm in
586	            this section.

588	        3.  Otherwise, increase pos by 1.

590	        4.  Return to step 2 in these substeps.

592	   8.   If s[pos] equals 0x21 (ASCII "!"):

594	        1.  Increase pos by 1.

596	        2.  If s[pos] equals 0x3E, then increase pos by 1 and jump back
597	            to the step labeled LOOP in the overall algorithm in this
598	            section.

600	        3.  Otherwise, return to step 1 in these substeps.

602	   9.   If s[pos] equals 0x3F (ASCII "?"):

604	        1.  Increase pos by 1.

606	        2.  If s[pos] and s[pos+1] equal 0x3F and 0x3E respectively,
607	            then increase pos by 1 and jump back to the step labeled
608	            LOOP in the overall algorithm in this section.

610	        3.  Otherwise, return to step 1 in these substeps.

612	   10.  Otherwise, if the octets in s starting at pos match any of the
613	        sequences of octets in the first column of the following table,
614	        then the user agent MUST follow the steps given in the
615	        corresponding cell in the second column of the same row.

617	 +----------------------+------------------------------------+---------+
618	 | Bytes in Hexadecimal | Requirement                        | Comment |
619	 +----------------------+------------------------------------+---------+
620	 | 72 73 73             | Let the sniffed-type be            | rss     |
621	 |                      | "application/rss+xml" and abort    |         |
622	 |                      | these steps.                       |         |
623	 +----------------------+------------------------------------+---------+
624	 | 66 65 65 64          | Let the sniffed-type be            | feed    |
625	 |                      | "application/atom+xml" and abort   |         |
626	 |                      | these steps.                       |         |
627	 +----------------------+------------------------------------+---------+
628	 | 72 64 66 3A 52 44 46 | Continue to the next step in this  | rdf:RDF |
629	 |                      | algorithm.                         |         |
630	 +----------------------+------------------------------------+---------+

632	        If none of the octet sequences above match the octets in s
633	        starting at pos, then let the sniffed-type be "text/html" and
634	        abort these steps.

636	   11.  Initialize RDF-flag to 0.

638	   12.  Initialize RSS-flag to 0.

640	   13.  If the octets with positions pos to pos+23 in s are exactly
641	        equal to 0x68, 0x74, 0x74, 0x70, 0x3A, 0x2F, 0x2F, 0x70, 0x75,
642	        0x72, 0x6C, 0x2E, 0x6F, 0x72, 0x67, 0x2F, 0x72, 0x73, 0x73,
643	        0x2F, 0x31, 0x2E, 0x30, 0x2F respectively (ASCII for
644	        "http://purl.org/rss/1.0/"), then:

646	        1.  Increase pos by 23.

648	        2.  Set RSS-flag to 1.

650	   14.  If the octets with positions pos to pos+42 in s are exactly
651	        equal to 0x68, 0x74, 0x74, 0x70, 0x3A, 0x2F, 0x2F, 0x77, 0x77,
652	        0x77, 0x2E, 0x77, 0x33, 0x2E, 0x6F, 0x72, 0x67, 0x2F, 0x31,
653	        0x39, 0x39, 0x39, 0x2F, 0x30, 0x32, 0x2F, 0x32, 0x32, 0x2D,
654	        0x72, 0x64, 0x66, 0x2D, 0x73, 0x79, 0x6E, 0x74, 0x61, 0x78,
655	        0x2D, 0x6E, 0x73, 0x23 respectively (ASCII for
656	        "http://www.w3.org/1999/02/22-rdf-syntax-ns#"), then:

658	        1.  Increase pos by 42.

660	        2.  Set RDF-flag to 1.

662	   15.  Increase pos by 1.

664	   16.  If RDF-flag is 1 and RSS-flag is 1, then let the sniffed-type be
665	        "application/rss+xml" and abort these steps.

667	   17.  If pos points beyond the end of the octet stream s, then
668	        continue to step 19 of this algorithm.

670	   18.  Jump back to step 13 of this algorithm.

672	   19.  Let the sniffed-type be "text/html" and abort these steps.

674	   For efficiency reasons, implementations might wish to implement this
675	   algorithm and the algorithm for detecting the character encoding of
676	   HTML documents in parallel.

678	8.  References

680	   [BarthCaballeroSong2009]
681	              Barth, A., Caballero, J., and D. Song, "Secure Content
682	              Sniffing for Web Browsers, or How to Stop Papers from
683	              Reviewing Themselves", 2009, <http://www.adambarth.com/
684	              papers/2009/barth-caballero-song.pdf>.

686	Authors' Addresses

688	   Adam Barth
689	   University of California, Berkeley

691	   Email: abarth@eecs.berkeley.edu
692	   URI:   http://www.adambarth.com/

694	   Ian Hickson
695	   Google, Inc.

697	   Email: ian@hixie.ch
698	   URI:   http://ln.hixie.ch/