idnits 2.17.1 draft-idempotency-header-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 17, 2020) is 1248 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 520 -- Looks like a reference, but probably isn't: '2' on line 520 -- Looks like a reference, but probably isn't: '3' on line 521 ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7231 (Obsoleted by RFC 9110) Summary: 2 errors (**), 0 flaws (~~), 1 warning (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Jena 3 Internet-Draft 4 Intended status: Standards Track S. Dalal 5 Expires: May 21, 2021 November 17, 2020 7 The Idempotency HTTP Header Field 8 draft-idempotency-header-00 10 Abstract 12 The "HTTP" Idempotency request header field can be used to carry 13 idempotency key in order to make non-idempotent "HTTP" methods such 14 as "POST" or "PATCH" fault-tolerant. 16 Status of This Memo 18 This Internet-Draft is submitted in full conformance with the 19 provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF). Note that other groups may also distribute 23 working documents as Internet-Drafts. The list of current Internet- 24 Drafts is at https://datatracker.ietf.org/drafts/current/. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet-Drafts as reference 29 material or to cite them other than as "work in progress." 31 This Internet-Draft will expire on May 21, 2021. 33 Copyright Notice 35 Copyright (c) 2020 IETF Trust and the persons identified as the 36 document authors. All rights reserved. 38 This document is subject to BCP 78 and the IETF Trust's Legal 39 Provisions Relating to IETF Documents 40 (https://trustee.ietf.org/license-info) in effect on the date of 41 publication of this document. Please review these documents 42 carefully, as they describe your rights and restrictions with respect 43 to this document. Code Components extracted from this document must 44 include Simplified BSD License text as described in Section 4.e of 45 the Trust Legal Provisions and are provided without warranty as 46 described in the Simplified BSD License. 48 Table of Contents 50 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 51 1.1. Notational Conventions . . . . . . . . . . . . . . . . . 3 52 2. The Idempotency HTTP Request Header Field . . . . . . . . . . 3 53 2.1. Syntax . . . . . . . . . . . . . . . . . . . . . . . . . 3 54 2.2. Uniqueness of Idempotency Key . . . . . . . . . . . . . . 4 55 2.3. Idempotency Key Validity and Expiry . . . . . . . . . . . 4 56 2.4. Idempotency Fingerprint . . . . . . . . . . . . . . . . . 4 57 2.5. Idempotency Enforcement Scenarios . . . . . . . . . . . . 4 58 2.6. Responsibilities . . . . . . . . . . . . . . . . . . . . 5 59 2.7. Error Scenarios . . . . . . . . . . . . . . . . . . . . . 5 60 3. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6 61 3.1. The Idempotency-Key HTTP Request Header Field . . . . . . 6 62 4. Implementation Status . . . . . . . . . . . . . . . . . . . . 7 63 4.1. Implementing the Concept . . . . . . . . . . . . . . . . 8 64 5. Security Considerations . . . . . . . . . . . . . . . . . . . 10 65 6. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 10 66 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 67 7.1. Normative References . . . . . . . . . . . . . . . . . . 11 68 7.2. Informative References . . . . . . . . . . . . . . . . . 11 69 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 12 70 Appendix B. Appendix . . . . . . . . . . . . . . . . . . . . . . 12 71 B.1. Appendix A. Imported ABNF . . . . . . . . . . . . . . . 12 72 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 12 74 1. Introduction 76 Idempotence is the property of certain operations in mathematics and 77 computer science whereby they can be applied multiple times without 78 changing the result beyond the initial application. It does not 79 matter if the operation is called only once, or 10s of times over. 80 The result "SHOULD" be the same. 82 Idempotency is important in building a fault-tolerant "HTTP API". An 83 "HTTP" request method is considered "idempotent" if the intended 84 effect on the server of multiple identical requests with that method 85 is the same as the effect for a single such request. [RFC7231] 86 defines methods "OPTIONS", "HEAD", "GET", "PUT" and "DELETE" as 87 idempotent. However, "POST" and "PATCH" methods are "NOT" 88 idempotent. 90 Suppose a client on "HTTP API" wants to create or update a resource 91 using "POST" method. Since "POST" is "NOT" an idempotent method, 92 calling it multiple times can result in duplication or wrong updates. 93 What would happen if you sent out the POST request to the server, but 94 you get a timeout? Is the resource actually created or updated? 95 Does the timeout happened during sending of the request to the 96 server, or while receiving the response on the client? Can we safely 97 retry again, or do we need to figure out first what has happened with 98 the resource? If "POST" was an idempotent method, we would not have 99 to answer such questions. We could safely resend a request until we 100 actually get a response back from the server. 102 For many use cases in "HTTP API", creation of duplicate records is a 103 severe problem from business perspective. For example, in Fintech 104 industry, duplicate records for requests involving any kind of 105 payment transaction on a financial account "MUST NOT" be allowed. In 106 other cases, processing of duplicate webhooks due to retries is not 107 warranted. 109 1.1. Notational Conventions 111 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 112 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 113 "OPTIONAL" in this document are to be interpreted as described in BCP 114 14 [RFC2119] [RFC8174] when, and only when, they appear in all 115 capitals, as shown here. 117 This specification uses the Augmented Backus-Naur Form (ABNF) 118 notation of [RFC5234] and includes, by reference, the IMF-fixdate 119 rule as defined in Section 7.1.1.1 of [RFC7231]. 121 The term "resource" is to be interpreted as defined in Section 2 of 122 [RFC7231], that is identified by an URI. 124 2. The Idempotency HTTP Request Header Field 126 An idempotency key is a unique value generated by the client which 127 the resource server uses to recognize subsequent retries of the same 128 request. The "Idempotency-Key" HTTP request header field carries 129 this key. 131 2.1. Syntax 133 The "Idempotency-Key" request header field describes 135 Idempotency-Key = idempotency-key-value 137 idempotency-key-value = opaque-value 138 opaque-value = DQUOTE *idempotencyvalue DQUOTE 139 idempotencyvalue = %x21 / %x23-7E / obs-text 140 ; VCHAR except double quotes, plus obs-text 142 Clients MUST NOT include more than one "Idempotency-Key" header field 143 in the same request. 145 The following example shows an idempotency key using UUID version 4 146 scheme: 148 Idempotency-Key: "8e03978e-40d5-43e8-bc93-6894a57f9324" 150 2.2. Uniqueness of Idempotency Key 152 The idempotency key that is supplied as part of every "POST" request 153 MUST be unique and can not be reused with another request with a 154 different request payload. 156 How to make the key unique is up to the client and it's agreed 157 protocol with the resource owner. It is "RECOMMENDED" that "UUID" or 158 a similar random identifier be used as the idempotency key. 160 2.3. Idempotency Key Validity and Expiry 162 The resource MAY enforce a time based idempotency keys, thus, be able 163 to purge or delete a key upon its expiry. The resource server SHOULD 164 publish expiration policy related documentation. 166 2.4. Idempotency Fingerprint 168 An idempotency fingerprint MAY be used in conjunction with with an 169 idempotency key to determine the uniqueness of a request and is 170 generated from request payload data. An idempotency fingerprint is 171 generated by the resource implementation. Idempotency Fingerprint 172 generation algorithm MAY use one of the following or similar 173 approaches to generate a fingerprint. 175 o Checksum of the entire request payload. 177 o Checksum of selected elements in the request payload. 179 o Field value match for each field in the request payload. 181 o Field value match for selected elements in the request payload. 183 o Request digest/signature. 185 2.5. Idempotency Enforcement Scenarios 187 o First time request (Idempotency Key and Idempotency Fingerprint 188 scenarios has not been seen) 190 The resource server SHOULD process the request normally and 191 respond with an appropriate response and status code. 193 o Duplicate request (Idempotency Key and Idempotency Fingerprint 194 scenarios has been seen) 196 Replay 198 The request was replayed after the original request completed. 199 The resource server MUST respond with the result of the previously 200 completed operation, success or an error. 202 Concurrent Request 204 The request was replayed before the original request completed. 205 The resource server MUST respond with a resource conflict error. 206 See ## Error Scenarios for details. 208 2.6. Responsibilities 210 Client 212 For the idempotent resource operations, the client MUST present a 213 unique idempotency key in Idempotency-Key request header field. 215 Resource Server 217 o Generate Idempotency Fingerprint when required. 219 o Check for idempotency under various scenarios including the ones 220 described earlier. 222 o Manage the lifecycle of the Idempotency Key. 224 o Publish idempotency related specification in relevant 225 documentation. 227 2.7. Error Scenarios 229 If the "Idempotency-Key" request header is missing for a documented 230 idempotent operation requiring this header, the resource server MUST 231 reply with an "HTTP" "400" status code with body containing a link 232 pointing to the relevant documentation. Alternately, using the 233 "HTTP" header "Link", client could be informed about the error too as 234 shown below. 236 HTTP/1.1 400 Bad Request 237 Link: ; 238 rel="describedby"; type="text/html" 240 If there is an attempt to reuse an idempotency key with a different 241 request payload, the resource server MUST reply with an "HTTP" "422" 242 status code with body containing a link pointing to the relevant 243 documentation. Using the "HTTP" header "Link", client could be 244 informed about the error as following. 246 HTTP/1.1 422 Unprocessable Entity 247 Link: ; 248 rel="describedby"; type="text/html" 250 If there is an attempt to reuse an idempotency key that is expired, 251 the resource server MUST reply with an "HTTP" "422" status code with 252 body containing a link pointing to the relevant documentation. Using 253 the "HTTP" header "Link", client could be informed about the error as 254 following. 256 HTTP/1.1 422 Unprocessable Entity 257 Link: ; 258 rel="describedby"; type="text/html" 260 If the request is replayed, while the original request is still 261 processing, the resource server MUST reply with an "HTTP" "409" 262 status code with body containing a link pointing to the relevant 263 documentation. Using the "HTTP" header "Link", client could be 264 informed about the error as following. 266 HTTP/1.1 409 Conflict 267 Link: ; 268 rel="describedby"; type="text/html" 270 For other errors, the resource MUST return the appropriate status 271 code and error message. 273 3. IANA Considerations 275 3.1. The Idempotency-Key HTTP Request Header Field 277 The "Idempotency-Key" request header should be added to the permanent 278 registry of message header fields (see [RFC3864]), taking into 279 account the guidelines given by HTTP/1.1 [RFC7231]. 281 Header Field Name: Idempotency-Key 283 Applicable Protocol: Hypertext Transfer Protocol (HTTP) 285 Status: Standard 287 Author: Jayadeba Jena, 288 Sanjay Dalal 290 Change controller: IETF 292 Specification document: this specification, 293 Section 2 "The Idempotency HTTP Request Header Field" 295 4. Implementation Status 297 Note to RFC Editor: Please remove this section before publication. 299 This section records the status of known implementations of the 300 protocol defined by this specification at the time of posting of this 301 Internet-Draft, and is based on a proposal described in [RFC7942]. 302 The description of implementations in this section is intended to 303 assist the IETF in its decision processes in progressing drafts to 304 RFCs. Please note that the listing of any individual implementation 305 here does not imply endorsement by the IETF. Furthermore, no effort 306 has been spent to verify the information presented here that was 307 supplied by IETF contributors. This is not intended as, and must not 308 be construed to be, a catalog of available implementations or their 309 features. Readers are advised to note that other implementations may 310 exist. 312 According to RFC 7942, "this will allow reviewers and working groups 313 to assign due consideration to documents that have the benefit of 314 running code, which may serve as evidence of valuable experimentation 315 and feedback that have made the implemented protocols more mature. 316 It is up to the individual working groups to use this information as 317 they see fit". 319 Organization: Stripe 321 o Description: Stripe uses custom HTTP header named "Idempotency- 322 Key" 324 o Reference: https://stripe.com/docs/idempotency 326 Organization: Adyen 328 o Description: Adyen uses custom HTTP header named "Idempotency-Key" 329 o Reference: https://docs.adyen.com/development-resources/api- 330 idempotency/ 332 Organization: Dwolla 334 o Description: Dwolla uses custom HTTP header named "Idempotency- 335 Key" 337 o Reference: https://docs.dwolla.com/ 339 Organization: Interledger 341 o Description: Interledger uses custom HTTP header named 342 "Idempotency-Key" 344 o Reference: https://github.com/interledger/ 346 Organization: WorldPay 348 o Description: WorldPay uses custom HTTP header named "Idempotency- 349 Key" 351 o Reference: https://developer.worldpay.com/docs/wpg/idempotency 353 Organization: Yandex 355 o Description: Yandex uses custom HTTP header named "Idempotency- 356 Key" 358 o Reference: https://cloud.yandex.com/docs/api-design- 359 guide/concepts/idempotency 361 4.1. Implementing the Concept 363 This is a list of implementations that implement the general concept, 364 but do so using different mechanisms: 366 Organization: Django 368 o Description: Django uses custom HTTP header named 369 "HTTP_IDEMPOTENCY_KEY" 371 o Reference: https://pypi.org/project/django-idempotency-key 373 Organization: Twilio 375 o Description: Twilio uses custom HTTP header named "I-Twilio- 376 Idempotency-Token" in webhooks 378 o Reference: https://www.twilio.com/docs/usage/webhooks/webhooks- 379 connection-overrides 381 Organization: PayPal 383 o Description: PayPal uses custom HTTP header named "PayPal-Request- 384 Id" 386 o Reference: https://developer.paypal.com/docs/business/develop/ 387 idempotency 389 Organization: RazorPay 391 o Description: RazorPay uses custom HTTP header named "X-Payout- 392 Idempotency" 394 o Reference: https://razorpay.com/docs/razorpayx/api/idempotency/ 396 Organization: OpenBanking 398 o Description: OpenBanking uses custom HTTP header called "x- 399 idempotency-key" 401 o Reference: https://openbankinguk.github.io/read-write-api- 402 site3/v3.1.6/profiles/read-write-data-api-profile.html#request- 403 headers 405 Organization: Square 407 o Description: To make an idempotent API call, Square recommends 408 adding a property named "idempotency_key" with a unique value in 409 the request body. 411 o Reference: https://developer.squareup.com/docs/build-basics/using- 412 rest-api 414 Organization: Google Standard Payments 416 o Description: Google Standard Payments API uses a property named 417 "requestId" in request body in order to provider idempotency in 418 various use cases. 420 o Reference: https://developers.google.com/standard-payments/ 421 payment-processor-service-api/rest/v1/TopLevel/capture 423 Organization: BBVA 424 o Description: BBVA Open Platform uses custom HTTP header called "X- 425 Unique-Transaction-ID" 427 o Reference: 428 https://bbvaopenplatform.com/apiReference/APIbasics/content/x- 429 unique-transaction-id 431 Organization: WebEngage 433 o Description: WebEngage uses custom HTTP header called "x-request- 434 id" to identify webhook POST requests uniquely to achieve events 435 idempotency. 437 o Reference: https://docs.webengage.com/docs/webhooks 439 5. Security Considerations 441 This section is meant to inform developers, information providers, 442 and users of known security concerns specific to the idempotency 443 keys. 445 For idempotent request handling, the resources MAY make use of the 446 value in the idempotency key to look up the idempotent request cache 447 such as a persistent store, for duplicate requests, matching the key. 448 If the resource does not validate the value of the idempotency key, 449 prior to performing the lookup, it MAY lead to various forms of 450 security attacks, compromising itself. To avoid such situations, the 451 resource SHOULD publish the expected format of the idempotency key 452 and always validate the value as per the published specification for 453 the key, before processing the request. 455 6. Examples 457 The first example shows an idempotency-key header field with key 458 value using UUID version 4 scheme: 460 Idempotency-Key: "8e03978e-40d5-43e8-bc93-6894a57f9324" 462 Second example shows an idempotency-key header field with key value 463 using some random string generator: 465 Idempotency-Key: "clkyoesmbgybucifusbbtdsbohtyuuwz" 467 7. References 468 7.1. Normative References 470 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 471 Requirement Levels", BCP 14, RFC 2119, 472 DOI 10.17487/RFC2119, March 1997, 473 . 475 [RFC3864] Klyne, G., Nottingham, M., and J. Mogul, "Registration 476 Procedures for Message Header Fields", BCP 90, RFC 3864, 477 DOI 10.17487/RFC3864, September 2004, 478 . 480 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 481 Specifications: ABNF", STD 68, RFC 5234, 482 DOI 10.17487/RFC5234, January 2008, 483 . 485 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 486 Protocol (HTTP/1.1): Message Syntax and Routing", 487 RFC 7230, DOI 10.17487/RFC7230, June 2014, 488 . 490 [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 491 Protocol (HTTP/1.1): Semantics and Content", RFC 7231, 492 DOI 10.17487/RFC7231, June 2014, 493 . 495 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 496 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 497 May 2017, . 499 7.2. Informative References 501 [RFC7942] Sheffer, Y. and A. Farrel, "Improving Awareness of Running 502 Code: The Implementation Status Section", BCP 205, 503 RFC 7942, DOI 10.17487/RFC7942, July 2016, 504 . 506 7.3. URIs 508 [1] https://github.com/paypal/api-standards/blob/master/ 509 patterns.md#idempotency 511 [2] https://stripe.com/docs/idempotency 513 [3] https://tools.ietf.org/html/draft-nottingham-http-poe-00 515 Appendix A. Acknowledgments 517 The authors would like to thank Mark Nottingham for his support for 518 this Internet Draft. We would like to acknowledge that this draft is 519 inspired by Idempotency related patterns described in API 520 documentation of PayPal [1] and Stripe [2] as well as Internet Draft 521 on POST Once Exactly [3] authored by Mark Nottingham. 523 The authors take all responsibility for errors and omissions. 525 Appendix B. Appendix 527 B.1. Appendix A. Imported ABNF 529 The following core rules are included by reference, as defined in 530 Appendix B.1 of [RFC5234]: ALPHA (letters), CR (carriage return), 531 CRLF (CR LF), CTL (controls), DIGIT (decimal 0-9), DQUOTE (double 532 quote), HEXDIG (hexadecimal 0-9/A-F/a-f), LF (line feed), OCTET (any 533 8-bit sequence of data), SP (space), and VCHAR (any visible US-ASCII 534 character). 536 The rules below are defined in [RFC7230]: 538 obs-text = 540 Authors' Addresses 542 Jayadeba Jena 544 Email: jjena@paypal.com 545 URI: https://github.com/jayadeba 547 Sanjay Dalal 549 Email: sanjay.dalal@cal.berkeley.edu 550 URI: https://github.com/sdatspun2