idnits 2.17.1 draft-nottingham-http-pipeline-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (August 10, 2010) is 5005 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-26) exists of draft-ietf-httpbis-p1-messaging-11 == Outdated reference: A later version (-20) exists of draft-ietf-httpbis-p3-payload-11 == Outdated reference: A later version (-26) exists of draft-ietf-httpbis-p6-cache-11 -- Obsolete informational reference (is this intentional?): RFC 2616 (Obsoleted by RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235) Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. Nottingham 3 Internet-Draft August 10, 2010 4 Intended status: Informational 5 Expires: February 11, 2011 7 Making HTTP Pipelining Usable on the Open Web 8 draft-nottingham-http-pipeline-00 10 Abstract 12 Pipelining was added to HTTP/1.1 as a means of improving the 13 performance of persistent connections in common cases. While it is 14 deployed in some limited circumstances, it is not widely used by 15 clients on the open Internet. This memo suggests some measures 16 designed to make it more possible for clients to reliably and safely 17 use HTTP pipelining in these situations. 19 This memo should be discussed on the ietf-http-wg@w3.org mailing 20 list, although it is not a work item of the HTTPbis WG. 22 Status of this Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at http://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on February 11, 2011. 39 Copyright Notice 41 Copyright (c) 2010 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 57 2. Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 3. HTTP Pipelining Issues . . . . . . . . . . . . . . . . . . . . 3 59 4. Discovering Faulty Proxies . . . . . . . . . . . . . . . . . . 4 60 5. Identifying Responses . . . . . . . . . . . . . . . . . . . . . 5 61 6. Signing Content for Integrity . . . . . . . . . . . . . . . . . 6 62 7. Hinting Pipelinable Content . . . . . . . . . . . . . . . . . . 6 63 8. Origin Server Considerations for Pipelining . . . . . . . . . . 7 64 9. User Agent Considerations for Pipelining . . . . . . . . . . . 7 65 10. Security Considerations . . . . . . . . . . . . . . . . . . . . 8 66 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 8 67 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 8 68 12.1. Normative References . . . . . . . . . . . . . . . . . . . 8 69 12.2. Informative References . . . . . . . . . . . . . . . . . . 8 70 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . . 8 71 Appendix B. Frequently Asked Questions . . . . . . . . . . . . . . 9 72 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 9 74 1. Introduction 76 HTTP/1.1 [RFC2616] added pipelining -- that is, the ability to have 77 more than one outstanding request on a connection at a particular 78 time -- to improve performance when many requests need to be made 79 (e.g., when an HTML page references several images). 81 Although not usable in all circumstances (e.g., POST, PUT and other 82 non-idempotent requests cannot be pipelined), for the common case of 83 Web browsing, pipelining seems at first like a broadly useful 84 improvement -- especially since the number of TCP connections 85 browsers and servers can use for a given interaction is limited, and 86 especially where there is noticeable latency present. 88 Indeed, in constrained applications of HTTP such as Subversion, 89 pipelining has been shown to improve end-user perceived latency 90 considerably. 92 However, pipelining is not broadly used on the Web today; while most 93 (but not all) servers and intermediaries support pipelining (to 94 varying degrees), only one major Web browser uses it in its default 95 configuration, and that implementation is reported to use a number of 96 proprietary heuristics to determine when it is safe to pipeline. 98 This memo characterises issues currently encountered in the use of 99 HTTP pipelining, and suggests the use of mechanisms that, when used 100 in concert, are designed to make its use more reliable and safe for 101 browsers. It does not propose large protocol changes (e.g., out-of- 102 order messages), but rather incremental improvements that can be 103 deployed within the confines of existing infrastructure. 105 2. Requirements 107 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 108 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 109 document are to be interpreted as described in [RFC2119]. 111 3. HTTP Pipelining Issues 113 Anecdotal evidence suggests there are a number of reasons why clients 114 don't use HTTP pipelining by default. Briefly, they are: 115 1. Server implementations may stall pipelined requests, or close 116 their connection. This is one of the most commonly cited 117 problems. 119 2. Server implementations may pipeline responses in the wrong order. 120 Some implementations mix up the order of pipelined responses; 121 e.g., when they hit an error state but don't "fill" the response 122 pipeline with a corresponding representation. 123 3. A few server implementations may corrupt pipelined responses. 124 It's been said that a very small number of implementations 125 actually interleave pipelined responses so that part of response 126 A appears in response B, which is both a security and 127 interoperability problem. 128 4. Clients don't have enough information about what is useful to 129 pipeline. A given response may take an inordinate amount of time 130 to generate, and/or be large enough to block subsequent 131 responses. Clients who pipeline may face worse performance if 132 they stack requests behind such an expensive request. 134 Note that here, "servers" can also include proxies and other 135 intermediaries. 137 The remainder of this memo proposes mechanisms that, together, can be 138 used to mitigate these issues. 140 4. Discovering Faulty Proxies 142 Issues specific to proxies are limited to the network infrastructure 143 currently used by the client, and it is reasonable to assume that 144 testing the infrastructure at the beginning of a session will 145 indicate how safe it is to pipeline while that infrastructure is in 146 use. 148 Such issues can be detected by sending pipelined requests to a known 149 server, and examining the responses for errors. 151 For example, if the ExampleBrowser implementation wishes to probe for 152 faulty proxies, it can send a series of requests to 153 "http://browser.example.com/pipeline-test/" and subresources. If the 154 bodies of the resulting responses deviate from those it expects in 155 any way, it is reasonable to assume that a faulty proxy is present, 156 and pipelining SHOULD NOT be used through it. 158 Typically, user agents will do this upon startup and changes in the 159 network, although they might periodically test to assure that a new 160 proxy hasn't been interposed. 162 Note that proxies aren't always configured explicitly. 164 5. Identifying Responses 166 HTTP relies on the context of the connection to associate a given 167 request with its intended response. In HTTP/1.0, this was a 168 reasonable assumption, since only one request could be outstanding at 169 a given time, and each request had exactly one response. 171 HTTP/1.1 made associating requests and responses in a given 172 connection more complex (and therefore fault-prone). Not only does 173 pipelining mean that multiple requests can be outstanding, but also 174 the 1xx series of response status codes introduce the possibility of 175 multiple response messages (syntactically) being associated with a 176 single request. 178 To improve the client's ability to correlate responses with their 179 requests and identify responses that are out of order (as well as 180 serve other potential use cases), this memo introduces the "Assoc- 181 Req" response header field. 183 Assoc-Req = "Assoc-Req" ":" OWS Assoc-Req-v 184 Assoc-Req-v = Method SP absolute-URI 186 The field-value of the Assoc-Req header field is the method and 187 effective request URI of the request associated with the response 188 that it appears in. The URI used MUST be generated using the 189 algorithm for finding the Effective Request URI in 190 [I-D.ietf-httpbis-p1-messaging]. The header field MUST NOT be 191 generated by proxies. 193 For example, given the following request over port 80: 195 GET /foo?it HTTP/1.1 196 Host: www.example.com 198 the appropriate Assoc-Req header field would be: 200 Assoc-Req: GET http://www.example.com/foo?it 202 Note that the Assoc-Req header field is not a perfectly reliable 203 identifier for the request associated with a response; for example, 204 it does not incorporate the selecting headers for content negotiation 205 [I-D.ietf-httpbis-p6-cache], nor does it differentiate request 206 bodies, when present. However, for the purposes of making pipelining 207 more reliable, it is sufficient. 209 Clients who wish to use the Assoc-Req response header field to aid in 210 identifying problems in pipelining can compare its values to those of 211 the request that they believe it to be associated with (based upon 212 HTTP's message parsing rules, defined in 213 [I-D.ietf-httpbis-p1-messaging]). If either the method or the URI 214 differ, it indicates that there may be a pipelining-related issue. 216 6. Signing Content for Integrity 218 Another means of protecting against server issues (whether proxy or 219 origin server) is to sign the response content for integrity, so that 220 any corruption becomes apparent. 222 One existing way to do this is to use the Content-MD5 header field 223 [I-D.ietf-httpbis-p3-payload]. 225 Clients who wish to use the Content-MD5 response header field to aid 226 in identifying corrupted content due to pipelining issues can compare 227 the hash to a calculated hash of the content. 229 In some circumstances, it may be impractical for the server to buffer 230 the response content in order to calculate a hash before sending it. 231 In these cases, the Content-MD5 response header can be send in an 232 HTTP trailer, provided that the connection is HTTP/1.1 from end to 233 end, and the client is able to process trailers. 235 Additional means of verifying HTTP response integrity may become 236 available in time. 238 7. Hinting Pipelinable Content 240 Finally, to assist clients in determining what requests are suitable 241 for pipelining, we define extensions to allow hinting by origin 242 servers. 244 Each of these hints indicates URLs that, when dereferenced, will 245 likely not incur significant latency on the server in generating the 246 response, nor significant latency on the network in transferring the 247 response. 249 What is "significant" is determined by the server. Clients will use 250 these hints to determine what request(s) it is safe to pipeline 251 something else after. 253 For example, if "http://example.com/a" is hinted, a client can be 254 more confident pipelining another request (e.g., to 255 "http://example.com/b") on the same request afterwards. 257 To allow flexibility and ease of administration, different kinds of 258 hints are defined: 259 o The "quick" link relation type [TBD: ref] can appear on individual 260 HTML elements such as "img", "script" and "link" to indicate that 261 the link they contain has low overhead. Additionally, it can be 262 used in the HTTP link header to indicate links within the response 263 in a format-neutral way. 264 o A document can indicate that all links from the indicated elements 265 have low overhead by using the HTML META "quick" element, with the 266 content indicating the element names that are "quick". For 267 example, "". 268 o [DISCUSS: is it worthwhile to define a /.well-known lookup 269 mechanism for quick links?] 271 8. Origin Server Considerations for Pipelining 273 To maximise the potential for request pipelining from clients that 274 support this specification, origin servers: 275 o SHOULD send the Assoc-Req response header field in all potentially 276 pipelinable responses (keeping in mind that downstream caches 277 might be serving responses in the future). 278 o SHOULD send the Content-MD5 response header (or trailer) field in 279 potentially pipelinable responses. 280 o SHOULD hint potentially pipelinable requests as outlined above. 282 9. User Agent Considerations for Pipelining 284 To take advantage of the server-side mechanisms defined in this 285 specification, user agents: 286 o SHOULD ascertain whether any proxies present (either configured or 287 interposed by interception) support pipelining by following the 288 protocol described above. If they do not, pipelining SHOULD NOT 289 be used. 290 o SHOULD check the Assoc-Req response header field-value, when 291 present, on all pipelined responses. 292 o SHOULD check the Content-MD5 response header (or trailer) field- 293 value, when present, on all pipelined responses. 294 o MAY use content hints for pipelining to assist in determining 295 whether to pipeline a given request. 297 Upon encountering an indication of pipelining problems with a 298 particular response (e.g., an incorrect Assoc-Req field-value, an 299 incorrect Content-MD5 field-value), user agents SHOULD discard the 300 response in question, all subsequent responses on the same 301 connection, close the connection. Unsatisfied requests can be 302 resubmitted, without pipelining, and the implementation can choose 303 not to use pipelining to the same server in the future. 305 10. Security Considerations 307 TBD 309 11. IANA Considerations 311 TBD 313 12. References 315 12.1. Normative References 317 [I-D.ietf-httpbis-p1-messaging] 318 Fielding, R., Gettys, J., Mogul, J., Nielsen, H., 319 Masinter, L., Leach, P., Berners-Lee, T., Lafon, Y., and 320 J. Reschke, "HTTP/1.1, part 1: URIs, Connections, and 321 Message Parsing", draft-ietf-httpbis-p1-messaging-11 (work 322 in progress), August 2010. 324 [I-D.ietf-httpbis-p3-payload] 325 Fielding, R., Gettys, J., Mogul, J., Nielsen, H., 326 Masinter, L., Leach, P., Berners-Lee, T., Lafon, Y., and 327 J. Reschke, "HTTP/1.1, part 3: Message Payload and Content 328 Negotiation", draft-ietf-httpbis-p3-payload-11 (work in 329 progress), August 2010. 331 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 332 Requirement Levels", BCP 14, RFC 2119, March 1997. 334 12.2. Informative References 336 [I-D.ietf-httpbis-p6-cache] 337 Fielding, R., Gettys, J., Mogul, J., Nielsen, H., 338 Masinter, L., Leach, P., Berners-Lee, T., Lafon, Y., 339 Nottingham, M., and J. Reschke, "HTTP/1.1, part 6: 340 Caching", draft-ietf-httpbis-p6-cache-11 (work in 341 progress), August 2010. 343 [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., 344 Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext 345 Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. 347 Appendix A. Acknowledgements 349 Thanks to Julian Reschke for help in defining the Assoc-Req field- 350 value. The author takes all responsibility for errors and omissions. 352 Appendix B. Frequently Asked Questions 354 Isn't full multiplexing better? 356 While "full" multiplexing is theoretically better, pipelining -- once 357 usable -- is adequate for almost all common Web browsing cases. 358 Since the browser needs to download HTML first, it has an opportunity 359 to receive hints about subsequent requests and pipeline them 360 appropriately. Likewise, by far the most common case for 361 multiplexing on the Web is when a large number of images and other 362 page assets need to be fetched with GET; a perfect use of pipelining, 363 provided that the client has enough information to avoid head-of-line 364 blocking. 366 Why not have the client generate a unique request identifier? 368 While in some ways this would be easier than the approach that the 369 Assoc-Req header takes, it would be more difficult to deploy, because 370 existing caching proxies wouldn't be able to serve the correct 371 identifier when using a cached response. 373 Author's Address 375 Mark Nottingham 377 Email: mnot@mnot.net 378 URI: http://www.mnot.net/