idnits 2.17.1 

draft-ietf-websec-mime-sniff-02.txt:

  Checking boilerplate required by RFC 5378 and the IETF Trust (see
  https://trustee.ietf.org/license-info):
  ----------------------------------------------------------------------------

  ** You're using the IETF Trust Provisions' Section 6.b License Notice from
     12 Sep 2009 rather than the newer Notice from 28 Dec 2009.  (See
     https://trustee.ietf.org/license-info/)


  Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
  ----------------------------------------------------------------------------

  == No 'Intended status' indicated for this document; assuming Proposed
     Standard


  Checking nits according to https://www.ietf.org/id-info/checklist :
  ----------------------------------------------------------------------------

  ** The document seems to lack a Security Considerations section.

  ** The document seems to lack an IANA Considerations section.  (See Section
     2.2 of https://www.ietf.org/id-info/checklist for how to handle the case
     when there are no actions for IANA.)

  ** The document seems to lack separate sections for Informative/Normative
     References.  All references will be assumed normative when checking for
     downward references.

  ** The document seems to lack a both a reference to RFC 2119 and the
     recommended RFC 2119 boilerplate, even if it appears to use RFC 2119
     keywords. 

     RFC 2119 keyword, line 115: '...ble, user agents SHOULD NOT employ a c...'
     RFC 2119 keyword, line 117: '..., the user agent SHOULD use the algori...'
     RFC 2119 keyword, line 127: '...ed as algorithms or specific steps MAY...'
     RFC 2119 keyword, line 147: '..., the user agent MUST use the textuall...'
     RFC 2119 keyword, line 175: '...  The user agent MUST use the followin...'
     (8 more instances...)


  Miscellaneous warnings:
  ----------------------------------------------------------------------------

  == The copyright year in the IETF Trust and authors Copyright Line does not
     match the current year

  -- The document date (February 4, 2011) is 4831 days in the past.  Is this
     intentional?

  -- Found something which looks like a code comment -- if you have code
     sections in the document, please surround them with '<CODE BEGINS>' and
     '<CODE ENDS>' lines.


  Checking references for intended status: Proposed Standard
  ----------------------------------------------------------------------------

     (See RFCs 3967 and 4897 for information about using normative references
     to lower-maturity documents in RFCs)

  == Missing Reference: 'RFC2046' is mentioned on line 171, but not defined

  -- Looks like a reference, but probably isn't: '0' on line 634

  -- Looks like a reference, but probably isn't: '1' on line 634

  -- Looks like a reference, but probably isn't: '2' on line 634

  -- Possible downref: Non-RFC (?) normative reference: ref.
     'BarthCaballeroSong2009'


     Summary: 5 errors (**), 0 flaws (~~), 3 warnings (==), 6 comments (--).

     Run idnits with the --verbose option for more detailed information about
     the items above.

--------------------------------------------------------------------------------


2	None                                                            A. Barth
3	Internet-Draft                                                I. Hickson
4	Expires: August 8, 2011                                     Google, Inc.
5	                                                        February 4, 2011

7	                          Media Type Sniffing
8	                    draft-ietf-websec-mime-sniff-02

10	Abstract

12	   Many web servers supply incorrect Content-Type header fields with
13	   their HTTP responses.  In order to be compatible with these servers,
14	   user agents consider the content of HTTP responses as well as the
15	   Content-Type header fields when determining the effective media type
16	   of the response.  This document describes an algorithm for
17	   determining the effective media type of HTTP responses that balances
18	   security and compatibility considerations.

20	   Please send feedback on this draft to websec@ietf.org.

22	Status of this Memo

24	   This Internet-Draft is submitted to IETF in full conformance with the
25	   provisions of BCP 78 and BCP 79.

27	   Internet-Drafts are working documents of the Internet Engineering
28	   Task Force (IETF), its areas, and its working groups.  Note that
29	   other groups may also distribute working documents as Internet-
30	   Drafts.

32	   Internet-Drafts are draft documents valid for a maximum of six months
33	   and may be updated, replaced, or obsoleted by other documents at any
34	   time.  It is inappropriate to use Internet-Drafts as reference
35	   material or to cite them other than as "work in progress."

37	   The list of current Internet-Drafts can be accessed at
38	   http://www.ietf.org/ietf/1id-abstracts.txt.

40	   The list of Internet-Draft Shadow Directories can be accessed at
41	   http://www.ietf.org/shadow.html.

43	   This Internet-Draft will expire on August 8, 2011.

45	Copyright Notice

47	   Copyright (c) 2011 IETF Trust and the persons identified as the
48	   document authors.  All rights reserved.

50	   This document is subject to BCP 78 and the IETF Trust's Legal
51	   Provisions Relating to IETF Documents
52	   (http://trustee.ietf.org/license-info) in effect on the date of
53	   publication of this document.  Please review these documents
54	   carefully, as they describe your rights and restrictions with respect
55	   to this document.  Code Components extracted from this document must
56	   include Simplified BSD License text as described in Section 4.e of
57	   the Trust Legal Provisions and are provided without warranty as
58	   described in the BSD License.

60	Table of Contents

62	   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
63	   2.  Metadata . . . . . . . . . . . . . . . . . . . . . . . . . . .  5
64	   3.  Web Pages  . . . . . . . . . . . . . . . . . . . . . . . . . .  6
65	   4.  Text or Binary . . . . . . . . . . . . . . . . . . . . . . . .  8
66	   5.  Unknown Type . . . . . . . . . . . . . . . . . . . . . . . . . 10
67	     5.1.  Signature for H.264  . . . . . . . . . . . . . . . . . . . 15
68	   6.  Image  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
69	   7.  Video  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
70	   8.  Fonts  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
71	   9.  Feed or HTML . . . . . . . . . . . . . . . . . . . . . . . . . 19
72	   10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22
73	   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23

75	1.  Introduction

77	   The HTTP Content-Type header field indicates the media type of an
78	   HTTP response.  However, many HTTP servers supply a Content-Type that
79	   does not match the actual contents of the response.  Historically,
80	   web browsers have tolerated these servers by examining the content of
81	   HTTP responses in addition to the Content-Type header field to
82	   determine the effective media type of the response.

84	   Without a clear specification of how to "sniff" the media type, each
85	   user agent implementor was forced to reverse engineer the behavior of
86	   the other user agents and to develop their own algorithm.  These
87	   divergent algorithms have lead to a lack of interoperability between
88	   user agents and to security issues when the server intends an HTTP
89	   response to be interpreted as one media type but some user agents
90	   interpret the responses as another media type.

92	   These security issues are most severe when an "honest" server lets
93	   potentially malicious users upload files and then serves the contents
94	   of those files with a low-privilege media type (such as text/plain or
95	   image/jpeg).  (Malicious servers, of course, can specify an arbitrary
96	   media type in the Content-Type header field.)  In the absence of
97	   media type sniffing, this user-generated content would not be
98	   interpreted as a high-privilege media type, such as text/html.
99	   However, if a user agent does interpret a low-privilege media type,
100	   such as image/gif, as a high-privilege media type, such as text/html,
101	   the user agent has created a privilege escalation vulnerability in
102	   the server.  For example, a malicious user might be able to leverage
103	   content sniffing to mount a cross-site script attack by including
104	   JavaScript code in the uploaded file that a user agent treats as
105	   text/html.

107	   This document describes a content sniffing algorithm that carefully
108	   balances the compatibility needs of user agent implementors with the
109	   security constraints.  The algorithm has been constructed with
110	   reference to content sniffing algorithms present in popular user
111	   agents, an extensive database of existing web content, and metrics
112	   collected from implementations deployed to a sizable number of users
113	   [BarthCaballeroSong2009].

115	   WARNING!  Whenever possible, user agents SHOULD NOT employ a content
116	   sniffing algorithm.  However, if a user agent does employ a content
117	   sniffing algorithm, the user agent SHOULD use the algorithm in this
118	   document because using a different content sniffing algorithm than
119	   servers expect causes security problems.  For example, if a server
120	   believes that the client will treat a contributed file as an image
121	   (and thus treat it as benign), but a user agent believes the content
122	   to be HTML (and thus privileged to execute any scripts contained
123	   therein), an attacker might be able to steal the user's
124	   authentication credentials and mount other cross-site scripting
125	   attacks.

127	   Conformance requirements phrased as algorithms or specific steps MAY
128	   be implemented in any manner, so long as the end result is
129	   equivalent.  (In particular, the algorithms defined in this
130	   specification are intended to be easy to follow, and not intended to
131	   be performant.)

133	2.  Metadata

135	   The explicit media type metadata information associated with sequence
136	   of octets depends on the protocol that was used to fetch the octets.

138	   For octets received via HTTP, the Content-Type HTTP header field, if
139	   present, indicates the media type.  Let the official-type be the
140	   media type indicted by the HTTP Content-Type header field, if
141	   present.  If the Content-Type header field is absent or if its value
142	   cannot be interpreted as a media type (e.g. because its value doesn't
143	   contain a U+002F SOLIDUS ('/') character), then there is no official-
144	   type.

146	      Note: If an HTTP response contains multiple Content-Type header
147	      fields, the user agent MUST use the textually last Content-Type
148	      header field to the official-type.  For example, if the last
149	      Content-Type header field contains the value "foo", then there is
150	      no official media type because "foo" cannot be interpreted as a
151	      media type (even if the HTTP response contains another Content-
152	      Type header field that could be interpreted as a media type).

154	   For octets fetched from the file system, user agents should use
155	   platform-specific conventions (e.g., operating system file extension/
156	   type mappings) to determine the official-type.

158	      Note: It is essential that file extensions are not used for
159	      determining the media type for octets fetched over HTTP because,
160	      in some cases, file extensions can be supplied by malicious
161	      parties.  For example, most PHP installations let the attacker
162	      append arbitrary path information to URLs (e.g.,
163	      http://example.com/foo.php/bar.html) and thereby determine the
164	      file extension.

166	   For octets fetched over some other protocols, e.g.  FTP, there is no
167	   type information.

169	   Note: Comparisons between media types, as defined by MIME
170	   specifications, are done in an ASCII case-insensitive manner.
171	   [RFC2046]

173	3.  Web Pages

175	   The user agent MUST use the following algorithm to determine the
176	   sniffed-type of a sequence of octets:

178	   1.  If the user agent is configured to strictly obey the official-
179	       type, then let the sniffed-type be the official-type and abort
180	       these steps.

182	   2.  If the octets were fetched via HTTP and there is an HTTP Content-
183	       Type header field and the value of the last such header field has
184	       octets that *exactly* match the octets contained in one of the
185	       following lines:

187	      +-------------------------------+--------------------------------+
188	      | Bytes in Hexadecimal          | Textual Representation         |
189	      +-------------------------------+--------------------------------+
190	      | 74 65 78 74 2f 70 6c 61 69 6e | text/plain                     |
191	      +-------------------------------+--------------------------------+
192	      | 74 65 78 74 2f 70 6c 61 69 6e | text/plain; charset=ISO-8859-1 |
193	      | 3b 20 63 68 61 72 73 65 74 3d |                                |
194	      | 49 53 4f 2d 38 38 35 39 2d 31 |                                |
195	      +-------------------------------+--------------------------------+
196	      | 74 65 78 74 2f 70 6c 61 69 6e | text/plain; charset=iso-8859-1 |
197	      | 3b 20 63 68 61 72 73 65 74 3d |                                |
198	      | 69 73 6f 2d 38 38 35 39 2d 31 |                                |
199	      +-------------------------------+--------------------------------+
200	      | 74 65 78 74 2f 70 6c 61 69 6e | text/plain; charset=UTF-8      |
201	      | 3b 20 63 68 61 72 73 65 74 3d |                                |
202	      | 55 54 46 2d 38                |                                |
203	      +-------------------------------+--------------------------------+

205	       ...then jump to the "text or binary" section below.

207	   3.  If there is no official-type, jump to the "unknown type" section
208	       below.

210	   4.  If the official-type is "unknown/unknown", "application/unknown",
211	       or "*/*", jump to the "unknown type" section below.

213	   5.  If the official-type ends in "+xml", or if it is either "text/
214	       xml" or "application/xml", then let the sniffed-type be the
215	       official-type and abort these steps.

217	   6.  If the official-type is an image type supported by the user agent
218	       (e.g., "image/png", "image/gif", "image/jpeg", etc), then jump to
219	       the "images" section below.

221	   7.  If the official-type is "text/html", then jump to the "feed or
222	       HTML" section below.

224	   8.  Let the sniffed-type be the official type.

226	4.  Text or Binary

228	   This section defines the *rules for distinguishing if a resource is
229	   text or binary*.

231	   1.  The user agent MAY wait for 512 or more octets be to arrive.

233	          Note: Waiting for 512 octets octets to arrive causes the text-
234	          or-binary algorithm to be deterministic for a given sequence
235	          of octets.  However, in some cases, the user agent might need
236	          to wait an arbitrary length of time for these octets to
237	          arrive.  User agents SHOULD wait for 512 octets to arrive,
238	          when feasible.

240	   2.  Let n be the smaller of either 512 or the number of octets that
241	       have already arrived.

243	   3.  If n is greater than or equal to 3, and the first 2 or 3 octets
244	       match one of the following octet sequences:

246	                   +----------------------+--------------+
247	                   | Bytes in Hexadecimal | Description  |
248	                   +----------------------+--------------+
249	                   | FE FF                | UTF-16BE BOM |
250	                   | FF FE                | UTF-16LE BOM |
251	                   | EF BB BF             | UTF-8 BOM    |
252	                   +----------------------+--------------+

254	       ...then let the sniffed-type be "text/plain" and abort these
255	       steps.

257	   4.  If none of the first n octets are binary data octets then let the
258	       sniffed-type be "text/plain" and abort these steps.

260	                         +-------------------------+
261	                         | Binary Data Byte Ranges |
262	                         +-------------------------+
263	                         | 0x00 -- 0x08            |
264	                         | 0x0B                    |
265	                         | 0x0E -- 0x1A            |
266	                         | 0x1C -- 0x1F            |
267	                         +-------------------------+

269	   5.  If the first octets match one of the octet sequences in the
270	       "pattern" column of the table in the "unknown type" section
271	       below, ignoring any rows whose cell in the "security" column says
272	       "scriptable" (or "n/a"), then let the sniffed-type be the type
273	       given in the corresponding cell in the "sniffed type" column on
274	       that row and abort these steps.

276	          WARNING!  It is critical that this step not ever return a
277	          scriptable type (e.g., text/html), because otherwise that
278	          would allow a privilege escalation attack.

280	   6.  Otherwise, let the sniffed-type be "application/octet-stream" and
281	       abort these steps.

283	5.  Unknown Type

285	   1.  The user agent MAY wait for 512 or more octets to arrive for the
286	       same reason as in the "text or binary" section above.

288	   2.  Let n be the smaller of either 512 or the number of octets that
289	       have already arrived.

291	   3.  For each row in the table below:

293	       *  If the row has no "WS" octets:

295	          1.  Let pattern-length be the length of the pattern.

297	          2.  If n is smaller than pattern-length then skip this row.

299	          3.  Apply the bit-wise "and" operator to the first pattern-
300	              length octets and the given mask, and let the result be
301	              the masked-data.

303	          4.  If the octets of the masked-data matches the given pattern
304	              octets exactly, then let the sniffed-type be the type
305	              given in the cell of the third column in that row and
306	              abort these steps.

308	       *  If the row has a "WS" octet or a "_>" octet:

310	          1.  Let index-pattern be an index into the mask and pattern
311	              octet strings of the row.

313	          2.  Let index-stream be an index into the octet stream being
314	              examined.

316	          3.  LOOP: If index-stream points beyond the end of the octet
317	              stream, then this row doesn't match and skip this row.

319	          4.  Examine the index-stream-th octet of the octet stream as
320	              follows:

322	              -  If the index-pattern-th octet of the pattern is a
323	                 normal hexadecimal octet and not a "WS" octet or a "_>"
324	                 octet:

326	                    If the bit-wise "and" operator, applied to the
327	                    index-stream-th octet of the stream and the index-
328	                    pattern-th octet of the mask, yield a value
329	                    different than the index-pattern-th octet of the
330	                    pattern, then skip this row.

332	                    Otherwise, increment index-pattern to the next octet
333	                    in the mask and pattern and index-stream to the next
334	                    octet in the octet stream.

336	              -  Otherwise, if the index-pattern-th octet of the pattern
337	                 is a "WS" octet:

339	                    "WS" means "whitespace", and allows insignificant
340	                    whitespace to be skipped when sniffing for a type
341	                    signature.

343	                    If the index-stream-th octet of the stream is one of
344	                    0x09 (ASCII TAB), 0x0A (ASCII LF), 0x0C (ASCII FF),
345	                    0x0D (ASCII CR), or 0x20 (ASCII space), then
346	                    increment only the index-stream to the next octet in
347	                    the octet stream.

349	                    Otherwise, increment only the index-pattern to the
350	                    next octet in the mask and pattern.

352	              -  Otherwise, if the index-pattern-th octet of the pattern
353	                 is a "_>" octet:

355	                    "_>" means "space-or-bracket", and allows HTML tag
356	                    names to terminate with either a space or a greater
357	                    than sign.

359	                    If index-stream-th octet of the stream different
360	                    than 0x20 (ASCII space) or 0x3E (ASCII ">"), then
361	                    skip this row.

363	                    Otherwise, increment index-pattern to the next octet
364	                    in the mask and pattern and index-stream to the next
365	                    octet in the octet stream.

367	          5.  If index-pattern does not point beyond the end of the mask
368	              and pattern octet strings, then jump back to the LOOP step
369	              in this algorithm.

371	          6.  Otherwise, let the sniffed-type be the type given in the
372	              cell of the third column in that row and abort these
373	              steps.

375	   4.  If the first n octets match the signature for H264 (as define in
376	       Section 5.1), then let the sniffed-type be video/H264 and abort
377	       these steps.

379	   5.  If none of the first n octets are binary data (as defined in the
380	       "text or binary" section), then let the sniffed-type be "text/
381	       plain" and abort these steps.

383	   6.  Otherwise, let the sniffed-type be "application/octet-stream" and
384	       abort these steps.

386	   The table used by the above algorithm is:

388	+-------------------+-------------------+-----------------+------------+
389	| Mask in Hex       | Pattern in Hex    | Sniffed Type    | Security   |
390	+-------------------+-------------------+-----------------+------------+
391	| FF FF FF DF DF DF | WS 3C 21 44 4F 43 | text/html       | Scriptable |
392	| DF DF DF DF FF DF | 54 59 50 45 20 48 |                 |            |
393	| DF DF DF FF       | 54 4D 4C _>       |                 |            |
394	| Comment: <!DOCTYPE HTML                                              |
395	+-------------------+-------------------+-----------------+------------+
396	| FF FF DF DF DF DF | WS 3C 48 54 4D 4C | text/html       | Scriptable |
397	| FF                | _>                |                 |            |
398	| Comment: <HTML                                                       |
399	+-------------------+-------------------+-----------------+------------+
400	| FF FF DF DF DF DF | WS 3C 48 45 41 44 | text/html       | Scriptable |
401	| FF                | _>                |                 |            |
402	| Comment: <HEAD                                                       |
403	+-------------------+-------------------+-----------------+------------+
404	| FF FF DF DF DF DF | WS 3C 53 43 52 49 | text/html       | Scriptable |
405	| DF DF FF          | 50 54 _>          |                 |            |
406	| Comment: <SCRIPT                                                     |
407	+-------------------+-------------------+-----------------+------------+
408	| FF FF DF DF DF DF | WS 3C 49 46 52 41 | text/html       | Scriptable |
409	| DF DF FF          | 4d 45 _>          |                 |            |
410	| Comment: <IFRAME                                                     |
411	+-------------------+-------------------+-----------------+------------+
412	| FF FF DF FF FF    | WS 3C 48 31 _>    | text/html       | Scriptable |
413	| Comment: <H1                                                         |
414	+-------------------+-------------------+-----------------+------------+
415	| FF FF DF DF DF FF | WS 3C 44 49 56 _> | text/html       | Scriptable |
416	| Comment: <DIV                                                        |
417	+-------------------+-------------------+-----------------+------------+
418	| FF FF DF DF DF DF | WS 3C 46 4f 4e 54 | text/html       | Scriptable |
419	| FF                | _>                |                 |            |
420	| Comment: <FONT                                                       |
421	+-------------------+-------------------+-----------------+------------+
422	| FF FF DF DF DF DF | WS 3C 54 41 42 4c | text/html       | Scriptable |
423	| DF FF             | 45 _>             |                 |            |
424	| Comment: <TABLE                                                      |
425	+-------------------+-------------------+-----------------+------------+
426	| FF FF DF FF       | WS 3C 41 _>       | text/html       | Scriptable |
427	| Comment: <A                                                          |
428	+-------------------+-------------------+-----------------+------------+
429	| FF FF DF DF DF DF | WS 3C 53 54 59 4c | text/html       | Scriptable |
430	| DF FF             | 45 _>             |                 |            |
431	| Comment: <STYLE                                                      |
432	+-------------------+-------------------+-----------------+------------+
433	| FF FF DF DF DF DF | WS 3C 54 49 54 4c | text/html       | Scriptable |
434	| DF FF             | 45 _>             |                 |            |
435	| Comment: <TITLE                                                      |
436	+-------------------+-------------------+-----------------+------------+
437	| FF FF DF FF       | WS 3C 42 _>       | text/html       | Scriptable |
438	| Comment: <B                                                          |
439	+-------------------+-------------------+-----------------+------------+
440	| FF FF DF DF DF DF | WS 3C 42 4f 44 59 | text/html       | Scriptable |
441	| FF                | _>                |                 |            |
442	| Comment: <BODY                                                       |
443	+-------------------+-------------------+-----------------+------------+
444	| FF FF DF DF FF    | WS 3C 42 52 _>    | text/html       | Scriptable |
445	| Comment: <BR                                                         |
446	+-------------------+-------------------+-----------------+------------+
447	| FF FF DF FF       | WS 3C 50 _>       | text/html       | Scriptable |
448	| Comment: <P                                                          |
449	+-------------------+-------------------+-----------------+------------+
450	| FF FF FF FF FF FF | WS 3C 21 2d 2d _> | text/html       | Scriptable |
451	| Comment: <!--                                                        |
452	+-------------------+-------------------+-----------------+------------+
453	| FF FF FF FF FF FF | WS 3C 3f 78 6d 6c | text/xml        | Scriptable |
454	| Comment: <?xml (Note the case sensitivity and lack of trailing _>)  |
455	+-------------------+-------------------+-----------------+------------+
456	| FF FF FF FF FF    | 25 50 44 46 2D    | application/pdf | Scriptable |
457	| Comment: The string "%PDF-", the PDF signature.                      |
458	+-------------------+-------------------+-----------------+------------+
459	| FF FF FF FF FF FF | 25 21 50 53 2D 41 | application/    | Safe       |
460	| FF FF FF FF FF    | 64 6F 62 65 2D    |      postscript |            |
461	| Comment: The string "%!PS-Adobe-", the PostScript signature.         |
462	+-------------------+-------------------+-----------------+------------+
463	| FF FF 00 00       | FE FF 00 00       | text/plain      | n/a        |
464	| Comment: UTF-16BE BOM                                                |
465	+-------------------+-------------------+-----------------+------------+
466	| FF FF 00 00       | FF FE 00 00       | text/plain      | n/a        |
467	| Comment: UTF-16LE BOM                                                |
468	+-------------------+-------------------+-----------------+------------+
469	| FF FF FF 00       | EF BB BF 00       | text/plain      | n/a        |
470	| Comment: UTF-8 BOM                                                   |
471	+-------------------+-------------------+-----------------+------------+
472	| FF FF FF FF FF FF | 47 49 46 38 37 61 | image/gif       | Safe       |
473	| Comment: The string "GIF87a", a GIF signature.                       |
474	+-------------------+-------------------+-----------------+------------+
475	| FF FF FF FF FF FF | 47 49 46 38 39 61 | image/gif       | Safe       |
476	| Comment: The string "GIF89a", a GIF signature.                       |
477	+-------------------+-------------------+-----------------+------------+
478	| FF FF FF FF FF FF | 89 50 4E 47 0D 0A | image/png       | Safe       |
479	| FF FF             | 1A 0A             |                 |            |
480	| Comment: The PNG signature.                                          |
481	+-------------------+-------------------+-----------------+------------+
482	| FF FF FF          | FF D8 FF          | image/jpeg      | Safe       |
483	| Comment: A JPEG SOI marker followed by a octet of another marker.    |
484	+-------------------+-------------------+-----------------+------------+
485	| FF FF             | 42 4D             | image/bmp       | Safe       |
486	| Comment: The string "BM", a BMP signature.                           |
487	+-------------------+-------------------+-----------------+------------+
488	| FF FF FF FF 00 00 | 52 49 46 46 00 00 | image/webp      | Safe       |
489	| 00 00 FF FF FF FF | 00 00 57 45 42 50 |                 |            |
490	| FF FF             | 56 50             |                 |            |
491	| Comment: "RIFF" followed by four bytes, followed by "WEBPVP".        |
492	+-------------------+-------------------+-----------------+------------+
493	| FF FF FF FF       | 00 00 01 00       | image/vnd.      | Safe       |
494	|                   |                   |  microsoft.icon |            |
495	| Comment: A Windows Icon signature.                                   |
496	+-------------------+-------------------+-----------------+------------+
497	| FF FF FF FF FF    | 4F 67 67 53 00    | application/ogg | Safe       |
498	| Comment: An Ogg audio or video signature.                     |
499	+-------------------+-------------------+-----------------+------------+
500	| FF FF FF FF 00 00 | 52 49 46 46 00 00 | audio/wave      | Safe       |
501	| 00 00 FF FF FF FF | 00 00 57 41 56 45 |                 |            |
502	| Comment: "RIFF" followed by four bytes, followed by "WAVE".          |
503	+-------------------+-------------------+-----------------+------------+
504	| FF FF FF FF       | 1A 45 DF A3       | video/webm      | Safe       |
505	| Comment: The WebM signature [TODO: Use more octets?]                 |
506	+-------------------+-------------------+-----------------+------------+
507	| FF FF FF FF FF FF | 52 61 72 20 1A 07 | application/    | Safe       |
508	| FF                | 00                | x-rar-compressed|            |
509	| Comment: A RAR archive.                                              |
510	+-------------------+-------------------+-----------------+------------+
511	| FF FF FF FF       | 50 4B 03 04       | application/zip | Safe       |
512	| Comment: A ZIP archive.                                              |
513	+-------------------+-------------------+-----------------+------------+
514	| FF FF FF          | 1F 8B 08          | application/    | Safe       |
515	|                   |                   |          x-gzip |            |
516	| Comment: A GZIP archive.                                             |
517	+-------------------+-------------------+-----------------+------------+
518	[TODO: MP3 audio.]

520	   User agents MAY support additional types if necessary, by implicitly
521	   adding to the above table.  However, user agents SHOULD NOT not use
522	   any other patterns for types already mentioned in the table above
523	   because this could then be used for privilege escalation (where,
524	   e.g., a server uses the above table to determine that content is not
525	   HTML and thus safe from cross-site scripting attacks, but then a user
526	   agent detects it as HTML anyway and allows script to execute).  In
527	   extending this table, user agents SHOULD NOT introduce any privilege
528	   escalation vulnerabilities.

530	   Note: The column marked "security" is used by the algorithm in the
531	   "text or binary" section, to avoid sniffing text/plain content as a
532	   type that can be used for a privilege escalation attack.

534	5.1.  Signature for H.264

536	   This section defines whether a sequence of n octets *matches the
537	   signature for H.264*.

539	   If n is less than 4, then the sequence does not match the signature
540	   for H264 and abort these steps.

542	   Let box-size be the value of the first four octets, interpreted as a
543	   32 bit unsigned, little-endian integer.

545	   If n is less than box-size or if box-size is not evenly divisible by
546	   4, then the sequence does not match the signature for H264 and abort
547	   these steps.

549	   If octets 5 through 8 (inclusive) of the sequence are not 0x66 0x74
550	   0x79 0x70 (the ASCII string "ftyp"), then the sequence does not match
551	   the signature for H264 and abort these steps.

553	   For each i from 2 to box-size/4 - 1 (inclusive):

555	   1.  If i is equal to 3, continue to the next i, if any.  (These
556	       octets correspond to the minor version number.)

558	   2.  If octets 4*i through 4*i + 3 (inclusive) of the sequence are
559	       0x6D 0x70 0x34 (the ASCII string "mp4"), then the sequence *does*
560	       match the signature for H264 and abort these steps.

562	   The sequence does not match the signature for H264.

564	6.  Image

566	   This section defines the *rules for sniffing images specifically*.

568	   If the official-type is "image/svg+xml", then let the sniffed-type be
569	   the official-type (an XML type) and abort these steps.

571	   If the first octets match one of the signatures in Section 5 for one
572	   of the following media types, then let the sniffed-type be the
573	   corresponding media type and abort these steps:

575	   o  image/gif

577	   o  image/png

579	   o  image/jpeg

581	   o  image/bmp

583	   o  image/vnd.microsoft.icon

585	   o  image/webp

587	   Otherwise, let the sniffed-type be the official-type and abort these
588	   steps.

590	7.  Video

592	   This section defines the *rules for sniffing videos specifically*.

594	   If the first octets match one of the signatures in Section 5 for one
595	   of the following media types, then let the sniffed-type be the
596	   corresponding media type and abort these steps:

598	   o  video/H264

600	   o  video/webm

602	   Otherwise, let the sniffed-type be the official-type and abort these
603	   steps.

605	8.  Fonts

607	   This section defines the *rules for sniffing fonts specifically*.

609	   TODO

611	   Otherwise, let the sniffed-type be the official-type and abort these
612	   steps.

614	9.  Feed or HTML

616	   1.   The user agent MAY wait for 512 or more octets to arrive for the
617	        same reason as in the "text or binary" section above.

619	   2.   Let s be the stream of octets, and let s[i] represent the octet
620	        in s with position i, treating s as zero-indexed (so the first
621	        octet is at i=0).

623	   3.   If at any point this algorithm requires the user agent to
624	        determine the value of a octet in s which has not yet arrived,
625	        or which is past the first 512 octets, or which is beyond the
626	        end of the octet stream, the algorithm stops and the sniffed-
627	        type is "text/html".

629	           Note: User agents are allowed, by the first step of this
630	           algorithm, to wait until the first 512 octets have arrived.

632	   4.   Initialize pos to 0.

634	   5.   If s[0] equals 0xEF, s[1] equals 0xBB, and s[2] equals 0xBF,
635	        then set pos to 3.  (This skips over a leading UTF-8 BOM, if
636	        any.)

638	   6.   LOOP: Examine s[pos].

640	        *  If it equals 0x09 (ASCII tab), 0x20 (ASCII space), 0x0A
641	           (ASCII LF), or 0x0D (ASCII CR)

643	              Increase pos by 1 and repeat this step.

645	        *  If it equals 0x3C (ASCII "<")

647	              Increase pos by 1 and go to the next step.

649	        *  If it is anything else

651	              Let the sniffed-type be "text/html" and abort these steps.

653	   7.   If the octets with positions pos to pos+2 in s are exactly equal
654	        to 0x21, 0x2D, 0x2D respectively (ASCII for "!--"), then:

656	        1.  Increase pos by 3.

658	        2.  If the octets with positions pos to pos+2 in s are exactly
659	            equal to 0x2D, 0x2D, 0x3E respectively (ASCII for "-->"),
660	            then increase pos by 3 and jump back to the previous step
661	            (the step labeled loop start) in the overall algorithm in
662	            this section.

664	        3.  Otherwise, increase pos by 1.

666	        4.  Return to step 2 in these substeps.

668	   8.   If s[pos] equals 0x21 (ASCII "!"):

670	        1.  Increase pos by 1.

672	        2.  If s[pos] equals 0x3E, then increase pos by 1 and jump back
673	            to the step labeled LOOP in the overall algorithm in this
674	            section.

676	        3.  Otherwise, return to step 1 in these substeps.

678	   9.   If s[pos] equals 0x3F (ASCII "?"):

680	        1.  Increase pos by 1.

682	        2.  If s[pos] and s[pos+1] equal 0x3F and 0x3E respectively,
683	            then increase pos by 1 and jump back to the step labeled
684	            LOOP in the overall algorithm in this section.

686	        3.  Otherwise, return to step 1 in these substeps.

688	   10.  Otherwise, if the octets in s starting at pos match any of the
689	        sequences of octets in the first column of the following table,
690	        then the user agent MUST follow the steps given in the
691	        corresponding cell in the second column of the same row.

693	 +----------------------+------------------------------------+---------+
694	 | Bytes in Hexadecimal | Requirement                        | Comment |
695	 +----------------------+------------------------------------+---------+
696	 | 72 73 73             | Let the sniffed-type be            | rss     |
697	 |                      | "application/rss+xml" and abort    |         |
698	 |                      | these steps.                       |         |
699	 +----------------------+------------------------------------+---------+
700	 | 66 65 65 64          | Let the sniffed-type be            | feed    |
701	 |                      | "application/atom+xml" and abort   |         |
702	 |                      | these steps.                       |         |
703	 +----------------------+------------------------------------+---------+
704	 | 72 64 66 3A 52 44 46 | Continue to the next step in this  | rdf:RDF |
705	 |                      | algorithm.                         |         |
706	 +----------------------+------------------------------------+---------+

708	        If none of the octet sequences above match the octets in s
709	        starting at pos, then let the sniffed-type be "text/html" and
710	        abort these steps.

712	   11.  Initialize RDF-flag to 0.

714	   12.  Initialize RSS-flag to 0.

716	   13.  If the octets with positions pos to pos+23 in s are exactly
717	        equal to 0x68, 0x74, 0x74, 0x70, 0x3A, 0x2F, 0x2F, 0x70, 0x75,
718	        0x72, 0x6C, 0x2E, 0x6F, 0x72, 0x67, 0x2F, 0x72, 0x73, 0x73,
719	        0x2F, 0x31, 0x2E, 0x30, 0x2F respectively (ASCII for
720	        "http://purl.org/rss/1.0/"), then:

722	        1.  Increase pos by 23.

724	        2.  Set RSS-flag to 1.

726	   14.  If the octets with positions pos to pos+42 in s are exactly
727	        equal to 0x68, 0x74, 0x74, 0x70, 0x3A, 0x2F, 0x2F, 0x77, 0x77,
728	        0x77, 0x2E, 0x77, 0x33, 0x2E, 0x6F, 0x72, 0x67, 0x2F, 0x31,
729	        0x39, 0x39, 0x39, 0x2F, 0x30, 0x32, 0x2F, 0x32, 0x32, 0x2D,
730	        0x72, 0x64, 0x66, 0x2D, 0x73, 0x79, 0x6E, 0x74, 0x61, 0x78,
731	        0x2D, 0x6E, 0x73, 0x23 respectively (ASCII for
732	        "http://www.w3.org/1999/02/22-rdf-syntax-ns#"), then:

734	        1.  Increase pos by 42.

736	        2.  Set RDF-flag to 1.

738	   15.  Increase pos by 1.

740	   16.  If RDF-flag is 1 and RSS-flag is 1, then let the sniffed-type be
741	        "application/rss+xml" and abort these steps.

743	   17.  If pos points beyond the end of the octet stream s, then
744	        continue to step 19 of this algorithm.

746	   18.  Jump back to step 13 of this algorithm.

748	   19.  Let the sniffed-type be "text/html" and abort these steps.

750	   For efficiency reasons, implementations might wish to implement this
751	   algorithm and the algorithm for detecting the character encoding of
752	   HTML documents in parallel.

754	10.  References

756	   [BarthCaballeroSong2009]
757	              Barth, A., Caballero, J., and D. Song, "Secure Content
758	              Sniffing for Web Browsers, or How to Stop Papers from
759	              Reviewing Themselves", 2009, <http://www.adambarth.com/
760	              papers/2009/barth-caballero-song.pdf>.

762	Authors' Addresses

764	   Adam Barth
765	   Google, Inc.

767	   Email: ietf@adambarth.com
768	   URI:   http://www.adambarth.com/

770	   Ian Hickson
771	   Google, Inc.

773	   Email: ian@hixie.ch
774	   URI:   http://ln.hixie.ch/