idnits 2.17.1 draft-frindell-httpbis-partial-post-replay-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC7230]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 28, 2019) is 1762 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'HTTP3' is defined on line 246, but no explicit reference was found in the text -- No information found for draft-ietf-quic-http-latest - is the name correct? -- Obsolete informational reference (is this intentional?): RFC 7230 (Obsoleted by RFC 9110, RFC 9112) -- Obsolete informational reference (is this intentional?): RFC 7540 (Obsoleted by RFC 9113) Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 httpbis Working Group A. Frindell 3 Internet-Draft Facebook 4 Intended status: Informational June 28, 2019 5 Expires: December 30, 2019 7 HTTP Partial POST Replay 8 draft-frindell-httpbis-partial-post-replay-00 10 Abstract 12 This memo introduces a method of exchanging HTTP [RFC7230] messages 13 between a web server and a cooperating intermediary - such as a 14 reverse proxy load balancer - that enables faster restarts for the 15 web server with minimal disruption for users. 17 Status of This Memo 19 This Internet-Draft is submitted in full conformance with the 20 provisions of BCP 78 and BCP 79. 22 Internet-Drafts are working documents of the Internet Engineering 23 Task Force (IETF). Note that other groups may also distribute 24 working documents as Internet-Drafts. The list of current Internet- 25 Drafts is at https://datatracker.ietf.org/drafts/current/. 27 Internet-Drafts are draft documents valid for a maximum of six months 28 and may be updated, replaced, or obsoleted by other documents at any 29 time. It is inappropriate to use Internet-Drafts as reference 30 material or to cite them other than as "work in progress." 32 This Internet-Draft will expire on December 30, 2019. 34 Copyright Notice 36 Copyright (c) 2019 IETF Trust and the persons identified as the 37 document authors. All rights reserved. 39 This document is subject to BCP 78 and the IETF Trust's Legal 40 Provisions Relating to IETF Documents 41 (https://trustee.ietf.org/license-info) in effect on the date of 42 publication of this document. Please review these documents 43 carefully, as they describe your rights and restrictions with respect 44 to this document. Code Components extracted from this document must 45 include Simplified BSD License text as described in Section 4.e of 46 the Trust Legal Provisions and are provided without warranty as 47 described in the Simplified BSD License. 49 1. Introduction 51 Web servers need to drain traffic periodically for configuration 52 changes, software updates and maintenance. As continuous deployment 53 becomes more common, the frequency of such events increases. When a 54 server shuts down, it chooses whether to let all existing requests 55 run to completion, or abort some or all in-progress requests. 56 Aborted requests lead to poor user experiences including error 57 messages or additional latency while the request is resent. Partial 58 POST Replay makes it possible to eliminate this class of errors by 59 handing off in-process requests to another server within a 60 deployment. 62 1.1. Conventions and Definitions 64 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 65 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 66 "OPTIONAL" in this document are to be interpreted as described in BCP 67 14 [RFC2119] [RFC8174] when, and only when, they appear in all 68 capitals, as shown here. 70 2. Partial POST Replay 72 This section describes the Partial POST Replay mechanism for handing 73 off a request with a partially transferred entity body to another 74 server instance. 76 2.1. Response Message 78 When the server begins restarting, it responds to any unprocessed 79 requests with incomplete entity bodies with a new 3xx status code 80 (TBD). The HTTP/1.1 status message is Partial POST Replay. Once 81 this status is sent the server MUST NOT process this request other 82 than is specified in this document. 84 The server MUST have prior knowledge that the intermediary supports 85 Partial POST Replay before sending the 3xx response. If a server 86 sends this response to an intermediary that does not understand it, 87 the response will likely be forwarded back to the client. 89 2.1.1. Response Headers 91 Each request header is echoed in the response message with the prefix 92 "Echo-". For example, the "User-Agent: Foo" request header would be 93 included in the response as "Echo-User-Agent: Foo". HTTP/2 [RFC7540] 94 and HTTP/3 {{?HTTP3} request pseudo-headers (beginning with ':') are 95 echoed in the response message with the prefix "Pseudo-Echo-", and 96 with the ':' removed. For example, ":path: /" is echoed as "pseudo- 97 echo-path: /". The server MUST NOT insert any Echo- or Pseudo-Echo 98 headers in the response if the corresponding header was not present 99 in the request. 101 Because there might be request body bytes in flight to the server 102 when the 3xx response is generated, the length of the response body 103 is unknown. The response SHOULD NOT include a "Content-Length" 104 header (but will include a "Echo-Content-Length" header, if the 105 request contained "Content-Length"). If the request protocol is 106 HTTP/1.1, the server SHOULD use chunked transfer encoding for the 107 response. 109 HTTP/1.1 server SHOULD include a "Connection: close" header in the 110 response to prevent the intermediary from reusing the connection for 111 a new request. HTTP/2 and HTTP/3 servers SHOULD emit a GOAWAY frame 112 on each open connection when shutdown is initiated. 114 2.2. Intermediary Processing 116 Intermediaries MUST track the number of body bytes forwarded to the 117 server for any request that could be replayed by the server. When an 118 intermediary receives a 3xx status code from the server, it stops 119 forwarding any new HTTP data from the client to this server. The 120 intermediary does not forward the 3xx response to the client, but 121 instead reconstructs the original HTTP request message from headers 122 in the response beginning with the "Echo-" or "Pseudo-Echo" prefixes. 123 Alternatively, if the intermediary retained a copy of the request it 124 MAY use that and discard the response headers. 126 The intermediary can choose to buffer the response before selecting a 127 new server, or can immediately select a new server and begin 128 forwarding the request there. When the entire replayed request body 129 has been sent to the new server, the intermediary can begin 130 forwarding new HTTP data from the client to the new server. 132 If the intermediary receives more body bytes from the server than it 133 forwarded, or if the response is terminated before receiving all 134 forwarded bytes, the intermediary MUST fail the request with a 5xx 135 status. 137 2.3. Original Request Termination 139 When the intermediary has received in the response body all of the 140 request bytes forwarded to the original server, it completes the 141 request message to the original server, according to the semantics of 142 the transport protocol: 144 o For HTTP/1.0, the intermediary half-closes the connection 145 o For HTTP/1.1, the intermediary sends the final chunk terminator, 146 or half-closes the connection if the request did not use chunked 147 transfer encoding. 149 o For HTTP/2, the intermediary sends a DATA frame with the 150 END_STREAM flag set on the request stream 152 o For HTTP/3, the intermediary sends a FIN on the request stream 154 When the server processes the end of the request, it completes the 155 response message according to the semantics of the transport 156 protocol. 158 Note that some HTTP server implementations treat the termination of 159 the request with fewer bytes than specified in the Content-Length 160 header as an error. Because all required information has been 161 transferred to the intermediary before this error occurs, the server 162 can abort the response and ignore the error without impacting the 163 final status of the request. 165 It is possible that the entire entity body was sent by the 166 intermediary before it received the Partial POST Replay status 167 message. In this case the intermediary will receive the entire 168 entity body in the response. 170 2.4. Preventing Loops 172 To prevent the intermediary from becoming stuck in an infinite 173 redirect loop, it SHOULD add a 'Partial-Post-Replay: 1' header 174 whenever forwarding to a new server. An intermediary that receives a 175 redirect response with more "Echo-Partial-Post-Replay" headers than 176 it supports SHOULD fail original request with a 5xx response. 178 3. Existing Solutions 180 There are several existing solutions to handling requests while 181 draining traffic from a web server, but each has drawbacks that 182 Partial POST Replay does not. 184 3.1. Drain Timeout 186 When servers stop accepting new connections, they often set a timeout 187 during which existing requests can continue processing. At the end 188 of the timeout, the server will abort any unfinished requests. 189 During this phase, the server is not operating at full capacity, and 190 requests that exceed the timeout are still terminated with error. 192 3.2. GOAWAY 194 HTTP/2 introduced the GOAWAY frame which a server can use to indicate 195 which requests will not be processed, and which can be safely retried 196 by the client. There are two problems with this mechanism. 198 First, the server cannot use this mechanism to refuse requests with 199 stream IDs lower than the highest stream ID it has already processed. 200 For example, if the server has received a partial request on stream 201 ID=3, but has already begun processing a request on stream ID=5, it 202 cannot send a GOAWAY with a Last-Stream-ID lower than 5. HTTP/2 does 203 not have a status code that indicates an individual request is 204 retryable 206 Second, an intermediary cannot seamlessly retry a POST request unless 207 it has buffered the entire request body. Buffering all request 208 bodies presents an enormous scalability challenge for intermediaries. 210 3.3. State Handover 212 Another possible technique is to pass state from a draining web 213 server to a new instance. Such deployments start a new instance to 214 handle new work in parallel with the instance that is shutting down. 215 This requires that the system have enough resources to run two 216 instances of the server simultaneously, for a potentially very long 217 time. 219 4. Security Considerations 221 An intermediary must trust the server to echo back the headers and 222 body of the original request. A malicious server could replay a 223 different request to the intermediary, who would then send it to 224 another server. The response to this forged request would be 225 interpreted as a response to the original request. 227 5. IANA Considerations 229 This document has no IANA actions. 231 6. References 233 6.1. Normative References 235 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 236 Requirement Levels", BCP 14, RFC 2119, 237 DOI 10.17487/RFC2119, March 1997, 238 . 240 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 241 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 242 May 2017, . 244 6.2. Informative References 246 [HTTP3] Bishop, M., Ed., "Hypertext Transfer Protocol Version 3 247 (HTTP/3)", draft-ietf-quic-http-latest (work in progress). 249 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 250 Protocol (HTTP/1.1): Message Syntax and Routing", 251 RFC 7230, DOI 10.17487/RFC7230, June 2014, 252 . 254 [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext 255 Transfer Protocol Version 2 (HTTP/2)", RFC 7540, 256 DOI 10.17487/RFC7540, May 2015, 257 . 259 Acknowledgments 261 This draft evolved from a feature developed at Facebook. Thanks to 262 Mohammad Husain, Woo Xie and David Langevin who worked on the initial 263 implementation and deployment of this feature. 265 Author's Address 267 Alan Frindell 268 Facebook 270 Email: afrind@fb.com