idnits 2.17.1 draft-nottingham-httpbis-retry-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 19 instances of too long lines in the document, the longest one being 10 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (February 1, 2017) is 2640 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Looks like a reference, but probably isn't: '4' on line 479 -- Looks like a reference, but probably isn't: '5' on line 484 -- Looks like a reference, but probably isn't: '1' on line 471 -- Looks like a reference, but probably isn't: '2' on line 473 -- Looks like a reference, but probably isn't: '3' on line 476 -- Looks like a reference, but probably isn't: '6' on line 532 -- Looks like a reference, but probably isn't: '7' on line 573 -- Looks like a reference, but probably isn't: '8' on line 633 -- Looks like a reference, but probably isn't: '9' on line 667 -- Looks like a reference, but probably isn't: '10' on line 690 -- Looks like a reference, but probably isn't: '11' on line 695 -- Looks like a reference, but probably isn't: '12' on line 715 -- Looks like a reference, but probably isn't: '13' on line 763 == Outdated reference: A later version (-34) exists of draft-ietf-quic-tls-01 ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7231 (Obsoleted by RFC 9110) == Outdated reference: A later version (-34) exists of draft-ietf-quic-transport-01 == Outdated reference: A later version (-28) exists of draft-ietf-tls-tls13-18 -- Obsolete informational reference (is this intentional?): RFC 7540 (Obsoleted by RFC 9113) Summary: 4 errors (**), 0 flaws (~~), 4 warnings (==), 16 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. Nottingham 3 Internet-Draft February 1, 2017 4 Intended status: Informational 5 Expires: August 5, 2017 7 Retrying HTTP Requests 8 draft-nottingham-httpbis-retry-01 10 Abstract 12 HTTP allows requests to be automatically retried under certain 13 circumstances. This draft explores how this is implemented, 14 requirements for similar functionality from other parts of the stack, 15 and potential future improvements. 17 Note to Readers 19 This draft is not intended to be published as an RFC. 21 The issues list for this draft can be found at 22 https://github.com/mnot/I-D/labels/httpbis-retry . 24 The most recent (often, unpublished) draft is at 25 https://mnot.github.io/I-D/httpbis-retry/ . 27 Recent changes are listed at https://github.com/mnot/I-D/commits/gh- 28 pages/httpbis-retry . 30 Status of This Memo 32 This Internet-Draft is submitted in full conformance with the 33 provisions of BCP 78 and BCP 79. 35 Internet-Drafts are working documents of the Internet Engineering 36 Task Force (IETF). Note that other groups may also distribute 37 working documents as Internet-Drafts. The list of current Internet- 38 Drafts is at http://datatracker.ietf.org/drafts/current/. 40 Internet-Drafts are draft documents valid for a maximum of six months 41 and may be updated, replaced, or obsoleted by other documents at any 42 time. It is inappropriate to use Internet-Drafts as reference 43 material or to cite them other than as "work in progress." 45 This Internet-Draft will expire on August 5, 2017. 47 Copyright Notice 49 Copyright (c) 2017 IETF Trust and the persons identified as the 50 document authors. All rights reserved. 52 This document is subject to BCP 78 and the IETF Trust's Legal 53 Provisions Relating to IETF Documents 54 (http://trustee.ietf.org/license-info) in effect on the date of 55 publication of this document. Please review these documents 56 carefully, as they describe your rights and restrictions with respect 57 to this document. Code Components extracted from this document must 58 include Simplified BSD License text as described in Section 4.e of 59 the Trust Legal Provisions and are provided without warranty as 60 described in the Simplified BSD License. 62 Table of Contents 64 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 65 1.1. Notational Conventions . . . . . . . . . . . . . . . . . 3 66 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 3 67 2.1. Retries and Replays: A Taxonomy of Repetition . . . . . . 3 68 2.2. What the Spec Says: Automatic Retries . . . . . . . . . . 4 69 2.3. What the Specs Say: Replay . . . . . . . . . . . . . . . 5 70 2.3.1. TCP Fast Open . . . . . . . . . . . . . . . . . . . . 5 71 2.3.2. TLS 1.3 . . . . . . . . . . . . . . . . . . . . . . . 5 72 2.3.3. QUIC . . . . . . . . . . . . . . . . . . . . . . . . 6 73 3. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 6 74 3.1. Automatic Retries In Practice . . . . . . . . . . . . . . 6 75 3.2. Replays Are Different . . . . . . . . . . . . . . . . . . 7 76 4. Possible Areas of Work . . . . . . . . . . . . . . . . . . . 8 77 4.1. Updating HTTP's Requirements for Retries . . . . . . . . 8 78 4.2. Protocol Extensions . . . . . . . . . . . . . . . . . . . 9 79 4.3. Feedback to Transport 0RTT Efforts . . . . . . . . . . . 9 80 5. Security Considerations . . . . . . . . . . . . . . . . . . . 9 81 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 9 82 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 83 7.1. Normative References . . . . . . . . . . . . . . . . . . 10 84 7.2. Informative References . . . . . . . . . . . . . . . . . 10 85 7.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 11 86 Appendix A. When Clients Retry . . . . . . . . . . . . . . . . . 11 87 A.1. Squid . . . . . . . . . . . . . . . . . . . . . . . . . . 11 88 A.2. Traffic Server . . . . . . . . . . . . . . . . . . . . . 12 89 A.3. Firefox . . . . . . . . . . . . . . . . . . . . . . . . . 14 90 A.4. Chromium . . . . . . . . . . . . . . . . . . . . . . . . 16 91 A.5. Curl . . . . . . . . . . . . . . . . . . . . . . . . . . 17 92 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 18 94 1. Introduction 96 One of the benefits of HTTP's well-defined method semantics is that 97 they allow failed requests to be retried, under certain 98 circumstances. 100 However, interest in extending, redefining or just clarifying HTTP's 101 retry semantics is increasing, for a number of reasons: 103 o Since HTTP/1.1's requirements were written, there has been a 104 substantial amount of experience deploying and using HTTP, leading 105 implementations to refine their behaviour, often diverging from 106 the specification. 108 o Likewise, changes such as HTTP/2 [RFC7540] might change the 109 underlying assumptions that these requirements were based upon. 111 o Emerging lower-layer developments such as TCP Fast Open [RFC7413], 112 TLS/1.3 [I-D.ietf-tls-tls13] and QUIC [I-D.ietf-quic-transport] 113 introduce the possibility of replayed requests in the beginning of 114 a connection, thanks to Zero Round Trip (0RT) modes. In some 115 ways, these are similar to retries - but not completely. 117 o Applications sometimes want requests to be retried by 118 infrastructure, but can't easily express them in a non-idempotent 119 request (such as GET). 121 This draft gives some background in Section 2, discusses aspects of 122 these issues in Section 3, suggesting possible areas of work in 123 Section 4, and cataloguing current implementation behaviours in 124 Appendix A. 126 1.1. Notational Conventions 128 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 129 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 130 document are to be interpreted as described in [RFC2119]. 132 2. Background 134 2.1. Retries and Replays: A Taxonomy of Repetition 136 In HTTP, there are three similar but separate phenomena that deserve 137 consideration for the purposes of this document: 139 1. *User Retries* happen when a user initiates an action that 140 results in a duplicate HTTP request message being emitted. For 141 example, a user retry might occur when a "reload" button is 142 pressed, a URL is typed in again, "return" is pressed in the URL 143 bar again, or a navigation link or form button is pressed twice 144 while still on screen. 146 2. *Automatic Retries* happen when an HTTP client implementation 147 resends a previous request message without user intervention or 148 initiation. This might happen when a GET request fails to return 149 a complete response, or when a connection drops before the 150 request is sent. Note that automatic retries can (and are) 151 performed both by user agents and intermediary clients. 153 3. *Replays* happen when the underlying transport units (e.g., TCP 154 packets, QUIC frames) containing a HTTP request message are re- 155 sent on the network *and* appear to be separate requests to the 156 downstream server, either automatically as part of transport 157 protocol operation, or by an attacker. The upstream HTTP client 158 might not have any indication that a replay has occurred. 160 Note that retries initiated by code shipped to the client by the 161 server (e.g., in JavaScript) occupy a grey area here. Because they 162 are not initiated by the generic HTTP client implementation itself, 163 we will consider them user retries for the time being. 165 Also, this document doesn't include transport layer loss recovery 166 (e.g., TCP retransmission). This is distinguished from replays 167 because the transport automatically suppresses duplicates. 169 2.2. What the Spec Says: Automatic Retries 171 [RFC7230], Section 6.3.1 allows HTTP requests to be retried in 172 certain circumstances: 174 When an inbound connection is closed prematurely, a client MAY 175 open a new connection and automatically retransmit an aborted 176 sequence of requests if all of those requests have idempotent 177 methods (Section 4.2.2 of [RFC7231]). A proxy MUST NOT 178 automatically retry non-idempotent requests. 180 A user agent MUST NOT automatically retry a request with a non- 181 idempotent method unless it has some means to know that the 182 request semantics are actually idempotent, regardless of the 183 method, or some means to detect that the original request was 184 never applied. For example, a user agent that knows (through 185 design or configuration) that a POST request to a given resource 186 is safe can repeat that request automatically. Likewise, a user 187 agent designed specifically to operate on a version control 188 repository might be able to recover from partial failure 189 conditions by checking the target resource revision(s) after a 190 failed connection, reverting or fixing any changes that were 191 partially applied, and then automatically retrying the requests 192 that failed. 194 A client SHOULD NOT automatically retry a failed automatic retry. 196 Note that the complete list of idempotent methods is maintained in 197 the IANA HTTP Method Registry [4]. 199 2.3. What the Specs Say: Replay 201 2.3.1. TCP Fast Open 203 [RFC7413], Section 6.3.1 addresses HTTP Request Replay with TCP Fast 204 Open: 206 While TFO is motivated by Web applications, the browser should not 207 use TFO to send requests in SYNs if those requests cannot tolerate 208 replays. One example is POST requests without application-layer 209 transaction protection (e.g., a unique identifier in the request 210 header). 212 On the other hand, TFO is particularly useful for GET requests. 213 GET request replay could happen across striped TCP connections: 214 after a server receives an HTTP request but before the ACKs of the 215 requests reach the browser, the browser may time out and retry the 216 same request on another (possibly new) TCP connection. This 217 differs from a TFO replay only in that the replay is initiated by 218 the browser, not by the TCP stack. 220 The same specification addresses HTTP over TLS in Section 6.3.2: 222 For Transport Layer Security (TLS) over TCP, it is safe and useful 223 to include a TLS client_hello in the SYN packet to save one RTT in 224 the TLS handshake. There is no concern about violating 225 idempotency. In particular, it can be used alone with the 226 speculative connection above. 228 2.3.2. TLS 1.3 230 [I-D.ietf-tls-tls13], Section 2.3 explains the properties of Zero-RTT 231 Data in TLS 1.3: 233 IMPORTANT NOTE: The security properties for 0-RTT data (regardless 234 of the cipher suite) are weaker than those for other kinds of TLS 235 data. Specifically: 237 1. This data is not forward secret, because it is encrypted 238 solely with the PSK. 240 2. There are no guarantees of non-replay between connections. 241 Unless the server takes special measures outside those 242 provided by TLS, the server has no guarantee that the same 243 0-RTT data was not transmitted on multiple 0-RTT connections 244 (See Section 4.2.6.2 for more details). This is especially 245 relevant if the data is authenticated either with TLS client 246 authentication or inside the application layer protocol. 247 However, 0-RTT data cannot be duplicated within a connection 248 (i.e., the server will not process the same data twice for the 249 same connection) and an attacker will not be able to make 250 0-RTT data appear to be 1-RTT data (because it is protected 251 with different keys.) 253 Section 4.2.6 defines a mechanism to limit the exposure to replay. 255 2.3.3. QUIC 257 [I-D.ietf-quic-tls] Section 7.2 says this about the risks of replay 258 during the 0RTT handshake: 260 If 0-RTT keys are available, the lack of replay protection means 261 that restrictions on their use are necessary to avoid replay 262 attacks on the protocol. 264 A client MUST only use 0-RTT keys to protect data that is 265 idempotent. A client MAY wish to apply additional restrictions on 266 what data it sends prior to the completion of the TLS handshake. 267 A client otherwise treats 0-RTT keys as equivalent to 1-RTT keys. 269 A client that receives an indication that its 0-RTT data has been 270 accepted by a server can send 0-RTT data until it receives all of 271 the server's handshake messages. A client SHOULD stop sending 272 0-RTT data if it receives an indication that 0-RTT data has been 273 rejected. 275 A server MUST NOT use 0-RTT keys to protect packets. 277 3. Discussion 279 3.1. Automatic Retries In Practice 281 In practice, it has been observed (see Appendix A) that some client 282 implementations (both user agent and intermediary) do automatically 283 retry requests. However, they do not do so consistently, and 284 arguably not in the spirit of the specification, unless this vague 285 catch-all: 287 some means to detect that the original request was never applied 289 is interpreted very broadly. 291 On the server side, it has been widely observed that content on the 292 Web doesn't always honour HTTP idemotency semantics, with many GET 293 requests incurring side effects, and with some sites even requiring 294 browsers to retry POST requests in order to properly interoperate. 296 Despite this situation, the Web seems to work reasonably well to date 297 (with notable exceptions [5]). 299 The status quo, therefore, is that no Web application can read HTTP's 300 retry requirements as a guarantee that any given request won't be 301 retried, even for methods that are not idempotent. As a result, 302 applications that care about avoiding duplicate requests need to 303 build a way to detect not only user retries but also automatic 304 retries into the application "above" HTTP itself. 306 3.2. Replays Are Different 308 TCP Fast Open [RFC7413], TLS/1.3 [I-D.ietf-tls-tls13] and QUIC 309 [I-D.ietf-quic-transport] all have mechanisms to carry application 310 data on the first packet sent by a client, to avoid the latency of 311 connection setup. 313 The request(s) in this first packet might be _replayed_, either 314 because the first packet (now carrying a HTTP request) is thought to 315 be lost and retransmitted, or because an attacker observes the packet 316 and sends a duplicate at some point in the future. 318 At first glance, it seems as if the idempotency semantics of HTTP 319 request methods could be used to determine what requests are suitable 320 for inclusion in the first packet of various 0RTT mechanisms being 321 discussed (as suggested by TCP Fast Open). For example, we could 322 disallow POST (and other non-idempotent methods) in 0RTT data. 324 Upon reflection, though, the observations above lead us to believe 325 that since any request might be retried (automatically or by users), 326 applications will still need to have a means of detecting duplicate 327 requests, thereby preventing side effects from replays as well as 328 retries. Thus, any HTTP request can be included in the first packet 329 of a 0RTT, despite the risk of replay. 331 Two types of attack specific to replayed HTTP requests need to be 332 taken into account, however: 334 1. A replay is a potential Denial of Service vector. An attacker 335 that can replay a request many times can probe for weaknesses in 336 retry protections, and can bring a server that needs to do any 337 substantial processing down. 339 2. An attacker might use a replayed request to leak information 340 about the response over time. If they can observe the encrypted 341 payload on the wire, they can infer the size of the response 342 (e.g., it might get bigger if the user's bank account has more in 343 it). 345 The first attack cannot be mitigated by HTTP; the 0RT mechanism 346 itself needs some transport-layer means of scoping the usability of 347 the first packet on a connection so that it cannot be reused broadly. 348 For example, this might be by time, or by network location. 350 The second attack is more difficult to mitigate; scoping the 351 usability of the first packet helps, but does not completely prevent 352 the attack. If the replayed request is state-changing, the 353 application's retry detection should kick in and prevent information 354 leakage (since the response will likely contain an error, instead of 355 the desired information). 357 If it is not (e.g., a GET), the information being targeted is 358 vulnerable as long as both the first packet and the credentials in 359 the request (if any) are valid. 361 4. Possible Areas of Work 363 4.1. Updating HTTP's Requirements for Retries 365 The currently language in [RFC7230] about retries is vague about the 366 conditions under which a request can be retried, leading to 367 significant variance in implementation behaviour. For example, it's 368 been observed that many automated clients fail under circumstances 369 when browsers succeed, because they do not retry in the same way. 371 As a result, more carefully specifying the conditions under which a 372 request can be retried would be helpful. Such work would need to 373 take into account varying conditions, such as: 375 o Connection closes 377 o TCP RST 378 o Connection timeouts 380 o Whether or not any part of the response has been received 382 o Whether or not it is the first request on the connection 384 o Variance due to use of HTTP/2, TLS/1.3, TCP Fast Open and QUIC. 386 Furthermore, readers might mistake the language in RFC7230 as 387 guaranteeing that some requests (e.g., POST) are never automatically 388 retried; this should be clarified. 390 4.2. Protocol Extensions 392 A number of mechanisms have been mooted at various times, e.g.: 394 o Adding a header to automatically retried requests, to aid de- 395 duplication by servers 397 o Defining a request header to by added by intermediaries when they 398 have received a request in a way that could have been replayed 400 o Defining a status code to allow servers to indicate that the 401 request needs to be sent in a way that can't be replayed 403 4.3. Feedback to Transport 0RTT Efforts 405 If the observations above hold, we should disabuse any notion that 406 HTTP method idempotency is a useful way to avoid problems with replay 407 attacks. Instead, we should encourage development of mechanisms to 408 mitigate the aspects of replay that are different than retries (e.g., 409 potential for DOS attacks). 411 5. Security Considerations 413 Yep. 415 6. Acknowledgements 417 Thanks to Brad Fitzpatrick, Leif Hedstrom, Subodh Iyengar, Amos 418 Jeffries, Patrick McManus, Matt Menke, Miroslav Ponec, Daniel 419 Stenberg and Martin Thomson for their input and feedback. 421 Thanks also to the participants in the 2016 HTTP Workshop for their 422 lively discussion of this topic. 424 7. References 426 7.1. Normative References 428 [I-D.ietf-quic-tls] 429 Thomson, M. and (. (Unknown), "Using Transport Layer 430 Security (TLS) to Secure QUIC", draft-ietf-quic-tls-01 431 (work in progress), January 2017. 433 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 434 Requirement Levels", BCP 14, RFC 2119, 435 DOI 10.17487/RFC2119, March 1997, 436 . 438 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 439 Protocol (HTTP/1.1): Message Syntax and Routing", 440 RFC 7230, DOI 10.17487/RFC7230, June 2014, 441 . 443 [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 444 Protocol (HTTP/1.1): Semantics and Content", RFC 7231, 445 DOI 10.17487/RFC7231, June 2014, 446 . 448 [RFC7413] Cheng, Y., Chu, J., Radhakrishnan, S., and A. Jain, "TCP 449 Fast Open", RFC 7413, DOI 10.17487/RFC7413, December 2014, 450 . 452 7.2. Informative References 454 [I-D.ietf-quic-transport] 455 Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed 456 and Secure Transport", draft-ietf-quic-transport-01 (work 457 in progress), January 2017. 459 [I-D.ietf-tls-tls13] 460 Rescorla, E., "The Transport Layer Security (TLS) Protocol 461 Version 1.3", draft-ietf-tls-tls13-18 (work in progress), 462 October 2016. 464 [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext 465 Transfer Protocol Version 2 (HTTP/2)", RFC 7540, 466 DOI 10.17487/RFC7540, May 2015, 467 . 469 7.3. URIs 471 [1] https://www.iana.org/assignments/http-methods/http-methods.xhtml 473 [2] https://signalvnoise.com/archives2/google_web_accelerator_hey_not 474 _so_fast_an_alert_for_web_app_designers.php 476 [3] http://bazaar.launchpad.net/~squid/squid/trunk/view/head:/src/ 477 FwdState.cc#L594 479 [4] https://git-wip- 480 us.apache.org/repos/asf?p=trafficserver.git;a=blob;f=proxy/http/H 481 ttpTransact.cc;h=8a1f5364d47654b118296a07a2a95284f119d84b;hb=HEAD 482 #l6408 484 [5] https://git-wip- 485 us.apache.org/repos/asf?p=trafficserver.git;a=blob;f=proxy/http/ 486 HttpTransact.cc;hb=48d7b25ba8a8229b0471d37cdaa6ef24cc634bb0#l3634 488 [6] http://mxr.mozilla.org/mozilla- 489 release/source/netwerk/protocol/http/nsHttpTransaction.cpp#938 491 [7] http://mxr.mozilla.org/mozilla- 492 release/source/netwerk/protocol/http/nsHttpRequestHead.cpp#67 494 [8] https://www.fxsitecompat.com/en-CA/docs/2016/post-request-fails- 495 on-certain-sites-showing-connection-reset-page/ 497 [9] https://chromium.googlesource.com/chromium/src.git/+/master/net/ 498 http/http_network_transaction.cc#1657 500 [10] https://github.com/curl/curl/blob/master/lib/transfer.c#L1892 502 Appendix A. When Clients Retry 504 In implementations, clients have been observed to retry requests in a 505 number of circumstances. 507 _Note: This section is intended to inform the discussion, not to be 508 published as a standard. If you have relevant information about 509 these or other implementations (open or closed), please get in 510 touch._ 512 A.1. Squid 514 Squid is a caching proxy server that retries requests that it 515 considers safe *or* idempotent, as long as there is not a request 516 body: 518 /// Whether we may try sending this request again after a failure. 519 bool 520 FwdState::checkRetriable() 521 { 522 // Optimize: A compliant proxy may retry PUTs, but Squid lacks the [rather 523 // complicated] code required to protect the PUT request body from being 524 // nibbled during the first try. Thus, Squid cannot retry some PUTs today. 525 if (request->body_pipe != NULL) 526 return false; 528 // RFC2616 9.1 Safe and Idempotent Methods 529 return (request->method.isHttpSafe() || request->method.isIdempotent()); 530 } 532 (source [6]) 534 Currently, it considers GET, HEAD, OPTIONS, REPORT, PROPFIND, SEARCH 535 and PRI to be safe, and GET, HEAD, PUT, DELETE, OPTIONS, TRACE, 536 PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, UNLOCK, and PRI to be 537 idempotent. 539 A.2. Traffic Server 541 Apache Traffic Server, a caching proxy server, ties retry-ability to 542 whether the request required a "tunnel" - i.e., forwarding the 543 request body to the next server. This is indicated by 544 "request_body_start", which is set when a POST tunnel is used. 546 // bool HttpTransact::is_request_retryable 547 // 548 // If we started a POST/PUT tunnel then we can 549 // not retry failed requests 550 // 551 bool 552 HttpTransact::is_request_retryable(State *s) 553 { 554 if (s->hdr_info.request_body_start == true) { 555 return false; 556 } 558 if (s->state_machine->plugin_tunnel_type != HTTP_NO_PLUGIN_TUNNEL) { 559 // API can override 560 if (s->state_machine->plugin_tunnel_type == HTTP_PLUGIN_AS_SERVER && 561 s->api_info.retry_intercept_failures == true) { 562 // This used to be an == comparison, which made no sense. Changed 563 // to be an assignment, hoping the state is correct. 564 s->state_machine->plugin_tunnel_type = HTTP_NO_PLUGIN_TUNNEL; 565 } else { 566 return false; 567 } 568 } 570 return true; 571 } 573 (source [7]) 575 When connected to an origin server, Traffic Server attempts to retry 576 under a number of failure conditions: 578 ///////////////////////////////////////////////////////////////////////// 579 // Name : handle_response_from_server 580 // Description: response is from the origin server 581 // 582 // Details : 583 // 584 // response from the origin server. one of three things can happen now. 585 // if the response is bad, then we can either retry (by first downgrading 586 // the request, maybe making it non-keepalive, etc.), or we can give up. 587 // the latter case is handled by handle_server_connection_not_open and 588 // sends an error response back to the client. if the response is good 589 // handle_forward_server_connection_open is called. 590 // 591 // 592 // Possible Next States From Here: 593 // 594 ///////////////////////////////////////////////////////////////////////// 595 void 596 HttpTransact::handle_response_from_server(State *s) 597 { 599 [...] 601 switch (s->current.state) { 602 case CONNECTION_ALIVE: 603 DebugTxn("http_trans", "[hrfs] connection alive"); 604 SET_VIA_STRING(VIA_DETAIL_SERVER_CONNECT, VIA_DETAIL_SERVER_SUCCESS); 605 s->current.server->clear_connect_fail(); 606 handle_forward_server_connection_open(s); 607 break; 609 [...] 611 case OPEN_RAW_ERROR: 612 /* fall through */ 613 case CONNECTION_ERROR: 614 /* fall through */ 615 case STATE_UNDEFINED: 616 /* fall through */ 617 case INACTIVE_TIMEOUT: 618 // Set to generic I/O error if not already set specifically. 619 if (!s->current.server->had_connect_fail()) 620 s->current.server->set_connect_fail(EIO); 622 if (is_server_negative_cached(s)) { 623 max_connect_retries = s->txn_conf->connect_attempts_max_retries_dead_server; 624 } else { 625 // server not yet negative cached - use default number of retries 626 max_connect_retries = s->txn_conf->connect_attempts_max_retries; 627 } 628 if (s->pCongestionEntry != NULL) 629 max_connect_retries = s->pCongestionEntry->connect_retries(); 631 if (is_request_retryable(s) && s->current.attempts < max_connect_retries) { 633 (source [8]) 635 A.3. Firefox 637 Firefox is a Web browser that retries under the following conditions: 639 // if the connection was reset or closed before we wrote any part of the 640 // request or if we wrote the request but didn't receive any part of the 641 // response and the connection was being reused, then we can (and really 642 // should) assume that we wrote to a stale connection and we must therefore 643 // repeat the request over a new connection. 644 // 645 // We have decided to retry not only in case of the reused connections, but 646 // all safe methods(bug 1236277). 647 // 648 // NOTE: the conditions under which we will automatically retry the HTTP 649 // request have to be carefully selected to avoid duplication of the 650 // request from the point-of-view of the server. such duplication could 651 // have dire consequences including repeated purchases, etc. 652 // 653 // NOTE: because of the way SSL proxy CONNECT is implemented, it is 654 // possible that the transaction may have received data without having 655 // sent any data. for this reason, mSendData == FALSE does not imply 656 // mReceivedData == FALSE. (see bug 203057 for more info.) 657 // 659 [...] 661 if (!mReceivedData && 662 ((mRequestHead && mRequestHead->IsSafeMethod()) || 663 !reallySentData || connReused)) { 664 // if restarting fails, then we must proceed to close the pipe, 665 // which will notify the channel that the transaction failed. 667 (source [9]) 669 ... and it considers GET, HEAD, OPTIONS, TRACE, PROPFIND, REPORT, and 670 SEARCH to be safe: 672 bool 673 nsHttpRequestHead::IsSafeMethod() const 674 { 675 // This code will need to be extended for new safe methods, otherwise 676 // they'll default to "not safe". 677 if (IsGet() || IsHead() || IsOptions() || IsTrace()) { 678 return true; 679 } 681 if (mParsedMethod != kMethod_Custom) { 682 return false; 683 } 685 return (!strcmp(mMethod.get(), "PROPFIND") || 686 !strcmp(mMethod.get(), "REPORT") || 687 !strcmp(mMethod.get(), "SEARCH")); 688 } 690 (source [10]) 692 Note that "connReused" is tested; if a connection has been used 693 before, Firefox will retry _any_ request, safe or not. A recent 694 change attempted to remove this behaviour, but it caused 695 compatibility problems [11], and is being backed out. 697 A.4. Chromium 699 Chromium is a Web browser that appears to retry any request when a 700 connection is broken, as long as it's successfully used the 701 connection before, and hasn't received any response headers yet: 703 bool HttpNetworkTransaction::ShouldResendRequest() const { 704 bool connection_is_proven = stream_->IsConnectionReused(); 705 bool has_received_headers = GetResponseHeaders() != NULL; 707 // NOTE: we resend a request only if we reused a keep-alive connection. 708 // This automatically prevents an infinite resend loop because we'll run 709 // out of the cached keep-alive connections eventually. 710 if (connection_is_proven && !has_received_headers) 711 return true; 712 return false; 713 } 715 (source [12]) 717 A.5. Curl 719 Curl is both a command-line client and widely-used library for HTTP. 720 Like Chromium, it will retry a request if the response hasn't 721 started. 723 CURLcode Curl_retry_request(struct connectdata *conn, 724 char **url) 725 { 726 struct Curl_easy *data = conn->data; 728 *url = NULL; 730 /* if we're talking upload, we can't do the checks below, unless the protocol 731 is HTTP as when uploading over HTTP we will still get a response */ 732 if(data->set.upload && 733 !(conn->handler->protocol&(PROTO_FAMILY_HTTP|CURLPROTO_RTSP))) 734 return CURLE_OK; 736 if((data->req.bytecount + data->req.headerbytecount == 0) && 737 conn->bits.reuse && 738 (data->set.rtspreq != RTSPREQ_RECEIVE)) { 739 /* We didn't get a single byte when we attempted to re-use a 740 connection. This might happen if the connection was left alive when we 741 were done using it before, but that was closed when we wanted to use it 742 again. Bad luck. Retry the same request on a fresh connect! */ 743 infof(conn->data, "Connection died, retrying a fresh connect\n"); 744 *url = strdup(conn->data->change.url); 745 if(!*url) 746 return CURLE_OUT_OF_MEMORY; 748 connclose(conn, "retry"); /* close this connection */ 749 conn->bits.retry = TRUE; /* mark this as a connection we're about 750 to retry. Marking it this way should 751 prevent i.e HTTP transfers to return 752 error just because nothing has been 753 transferred! */ 755 if(conn->handler->protocol&PROTO_FAMILY_HTTP) { 756 struct HTTP *http = data->req.protop; 757 if(http->writebytecount) 758 return Curl_readrewind(conn); 759 } 760 } 761 return CURLE_OK; 762 } 763 (source [13]) 765 Author's Address 767 Mark Nottingham 769 Email: mnot@mnot.net 770 URI: https://www.mnot.net/