idnits 2.17.1 draft-idempotency-header-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 23, 2020) is 1248 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 537 -- Looks like a reference, but probably isn't: '2' on line 537 -- Looks like a reference, but probably isn't: '3' on line 538 ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7231 (Obsoleted by RFC 9110) Summary: 2 errors (**), 0 flaws (~~), 1 warning (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Jena 3 Internet-Draft PayPal, Inc. 4 Intended status: Standards Track S. Dalal 5 Expires: May 27, 2021 November 23, 2020 7 The Idempotency HTTP Header Field 8 draft-idempotency-header-01 10 Abstract 12 The "HTTP" Idempotency request header field can be used to carry 13 idempotency key in order to make non-idempotent "HTTP" methods such 14 as "POST" or "PATCH" fault-tolerant. 16 Status of This Memo 18 This Internet-Draft is submitted in full conformance with the 19 provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF). Note that other groups may also distribute 23 working documents as Internet-Drafts. The list of current Internet- 24 Drafts is at https://datatracker.ietf.org/drafts/current/. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet-Drafts as reference 29 material or to cite them other than as "work in progress." 31 This Internet-Draft will expire on May 27, 2021. 33 Copyright Notice 35 Copyright (c) 2020 IETF Trust and the persons identified as the 36 document authors. All rights reserved. 38 This document is subject to BCP 78 and the IETF Trust's Legal 39 Provisions Relating to IETF Documents 40 (https://trustee.ietf.org/license-info) in effect on the date of 41 publication of this document. Please review these documents 42 carefully, as they describe your rights and restrictions with respect 43 to this document. Code Components extracted from this document must 44 include Simplified BSD License text as described in Section 4.e of 45 the Trust Legal Provisions and are provided without warranty as 46 described in the Simplified BSD License. 48 Table of Contents 50 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 51 1.1. Notational Conventions . . . . . . . . . . . . . . . . . 3 52 2. The Idempotency HTTP Request Header Field . . . . . . . . . . 3 53 2.1. Syntax . . . . . . . . . . . . . . . . . . . . . . . . . 3 54 2.2. Uniqueness of Idempotency Key . . . . . . . . . . . . . . 4 55 2.3. Idempotency Key Validity and Expiry . . . . . . . . . . . 4 56 2.4. Idempotency Fingerprint . . . . . . . . . . . . . . . . . 4 57 2.5. Responsibilities . . . . . . . . . . . . . . . . . . . . 4 58 2.6. Idempotency Enforcement Scenarios . . . . . . . . . . . . 5 59 2.7. Error Scenarios . . . . . . . . . . . . . . . . . . . . . 6 60 3. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6 61 3.1. The Idempotency-Key HTTP Request Header Field . . . . . . 6 62 4. Implementation Status . . . . . . . . . . . . . . . . . . . . 7 63 4.1. Implementing the Concept . . . . . . . . . . . . . . . . 8 64 5. Security Considerations . . . . . . . . . . . . . . . . . . . 10 65 6. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 10 66 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 67 7.1. Normative References . . . . . . . . . . . . . . . . . . 11 68 7.2. Informative References . . . . . . . . . . . . . . . . . 11 69 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 12 70 Appendix B. Appendix . . . . . . . . . . . . . . . . . . . . . . 12 71 B.1. Appendix A. Imported ABNF . . . . . . . . . . . . . . . 12 72 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 12 74 1. Introduction 76 Idempotence is the property of certain operations in mathematics and 77 computer science whereby they can be applied multiple times without 78 changing the result beyond the initial application. It does not 79 matter if the operation is called only once, or 10s of times over. 80 The result "SHOULD" be the same. 82 Idempotency is important in building a fault-tolerant "HTTP API". An 83 "HTTP" request method is considered "idempotent" if the intended 84 effect on the server of multiple identical requests with that method 85 is the same as the effect for a single such request. According to 86 [RFC7231], "HTTP" methods "OPTIONS", "HEAD", "GET", "PUT" and 87 "DELETE" are idempotent while methods "POST" and "PATCH" are not. 89 Let's say a client of an "HTTP API" wants to create (or update) a 90 resource using a "POST" method. Since "POST" is "NOT" an idempotent 91 method, calling it multiple times can result in duplication or wrong 92 updates. Consider a scenario where the client sent a "POST" request 93 to the server, but it got a timeout. Following questions arise : Is 94 the resource actually created (or updated)? Did the timeout occur 95 during sending of the request, or when receiving of the response? 96 Can the client safely retry the request, or does it need to figure 97 out what happened in the first place? If "POST" had been an 98 idempotent method, such questions may not arise. Client would safely 99 retry a request until it actually gets a valid response from the 100 server. 102 For many use cases of "HTTP API", duplicate resource is a severe 103 problem from business perspective. For example, duplicate records 104 for requests involving any kind of money transfer "MUST NOT" be 105 allowed. In other cases, processing of duplicate webhook delivery is 106 not expected. 108 1.1. Notational Conventions 110 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 111 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 112 "OPTIONAL" in this document are to be interpreted as described in BCP 113 14 [RFC2119] [RFC8174] when, and only when, they appear in all 114 capitals, as shown here. 116 This specification uses the Augmented Backus-Naur Form (ABNF) 117 notation of [RFC5234] and includes, by reference, the IMF-fixdate 118 rule as defined in Section 7.1.1.1 of [RFC7231]. 120 The term "resource" is to be interpreted as defined in Section 2 of 121 [RFC7231], that is identified by an URI. The term "resource server" 122 is to be interpreted as "origin server" as defined in Section 3 of 123 [RFC7231]. 125 2. The Idempotency HTTP Request Header Field 127 An idempotency key is a unique value generated by the client which 128 the resource server uses to recognize subsequent retries of the same 129 request. The "Idempotency-Key" HTTP request header field carries 130 this key. 132 2.1. Syntax 134 The "Idempotency-Key" request header field describes 136 Idempotency-Key = idempotency-key-value 138 idempotency-key-value = opaque-value 139 opaque-value = DQUOTE *idempotencyvalue DQUOTE 140 idempotencyvalue = %x21 / %x23-7E / obs-text 141 ; VCHAR except double quotes, plus obs-text 143 Clients MUST NOT include more than one "Idempotency-Key" header field 144 in the same request. 146 The following example shows an idempotency key using "UUID" 147 [RFC4122]: 149 Idempotency-Key: "8e03978e-40d5-43e8-bc93-6894a57f9324" 151 2.2. Uniqueness of Idempotency Key 153 The idempotency key that is supplied as part of every "POST" request 154 MUST be unique and "MUST" not be reused with another request with a 155 different request payload. 157 Uniqueness of the key "MUST" be defined by the resource owner and 158 "MUST" be implemented by the clients of the resource server. It is 159 "RECOMMENDED" that "UUID" [RFC4122] or a similar random identifier be 160 used as an idempotency key. 162 2.3. Idempotency Key Validity and Expiry 164 The resource MAY enforce time based idempotency keys, thus, be able 165 to purge or delete a key upon its expiry. The resource server SHOULD 166 define such expiration policy and publish in related documentation. 168 2.4. Idempotency Fingerprint 170 An idempotency fingerprint MAY be used in conjunction with an 171 idempotency key to determine the uniqueness of a request. Such a 172 fingerprint is generated from request payload data by the resource 173 server. An idempotency fingerprint generation algorithm MAY use one 174 of the following or similar approaches to create a fingerprint. 176 o Checksum of the entire request payload. 178 o Checksum of selected element(s) in the request payload. 180 o Field value match for each field in the request payload. 182 o Field value match for selected element(s) in the request payload. 184 o Request digest/signature. 186 2.5. Responsibilities 188 Client 189 Clients of "HTTP API" requiring idempotency, SHOULD understand the 190 idempotency related requirements as published by the server and use 191 appropriate algorithm to generate idempotency keys. 193 For each request, client SHOULD 195 o Send a unique idempotency key in the HTTP "Idempotency-Key" 196 request header field. 198 Resource Server 200 Resource server MUST publish idempotency related specification. This 201 specification MUST include expiration related policy if applicable. 202 Server is responsible for managing the lifecycle of the idempotency 203 key. 205 For each request, server SHOULD 207 o Identify idempotency key from the HTTP "Idempotency-Key" request 208 header field. 210 o Generate idempotency fingerprint if required. 212 o Check for idempotency considering various scenarios including the 213 ones described in section below. 215 2.6. Idempotency Enforcement Scenarios 217 o First time request (idempotency key or fingerprint has not been 218 seen) 220 The resource server SHOULD process the request normally and 221 respond with an appropriate response and status code. 223 o Duplicate request (idempotency key or fingerprint has been seen) 225 Retry 227 The request was retried after the original request completed. The 228 resource server MUST respond with the result of the previously 229 completed operation, success or an error. 231 Concurrent Request 233 The request was retried before the original request completed. 234 The resource server MUST respond with a resource conflict error. 235 See Error Scenarios for details. 237 2.7. Error Scenarios 239 If the "Idempotency-Key" request header is missing for a documented 240 idempotent operation requiring this header, the resource server MUST 241 reply with an "HTTP" "400" status code with body containing a link 242 pointing to relevant documentation. Alternately, using the "HTTP" 243 header "Link", the client can be informed about the error as shown 244 below. 246 HTTP/1.1 400 Bad Request 247 Link: ; 248 rel="describedby"; type="text/html" 250 If there is an attempt to reuse an idempotency key with a different 251 request payload, the resource server MUST reply with a "HTTP" "422" 252 status code with body containing a link pointing to relevant 253 documentation. The status code "422" is defined in Section 11.2 of 254 [RFC4918]. The server can also inform the client by using the "HTTP" 255 header "Link" as shown below. 257 HTTP/1.1 422 Unprocessable Entity 258 Link: ; 259 rel="describedby"; type="text/html" 261 If the request is retried, while the original request is still being 262 processed, the resource server MUST reply with an "HTTP" "409" status 263 code with body containing a link or the "HTTP" header "Link" pointing 264 to the relevant documentation. 266 HTTP/1.1 409 Conflict 267 Link: ; 268 rel="describedby"; type="text/html" 270 For other errors, the resource MUST return the appropriate status 271 code and error message. 273 3. IANA Considerations 275 3.1. The Idempotency-Key HTTP Request Header Field 277 The "Idempotency-Key" request header should be added to the permanent 278 registry of message header fields (see [RFC3864]), taking into 279 account the guidelines given by HTTP/1.1 [RFC7231]. 281 Header Field Name: Idempotency-Key 283 Applicable Protocol: Hypertext Transfer Protocol (HTTP) 285 Status: Standard 287 Authors: 288 Jayadeba Jena 289 Email: jjena@paypal.com 291 Sanjay Dalal 292 Email: sanjay.dalal@cal.berkeley.edu 294 Change controller: IETF 296 Specification document: this specification, 297 Section 2 "The Idempotency HTTP Request Header Field" 299 4. Implementation Status 301 Note to RFC Editor: Please remove this section before publication. 303 This section records the status of known implementations of the 304 protocol defined by this specification at the time of posting of this 305 Internet-Draft, and is based on a proposal described in [RFC7942]. 306 The description of implementations in this section is intended to 307 assist the IETF in its decision processes in progressing drafts to 308 RFCs. Please note that the listing of any individual implementation 309 here does not imply endorsement by the IETF. Furthermore, no effort 310 has been spent to verify the information presented here that was 311 supplied by IETF contributors. This is not intended as, and must not 312 be construed to be, a catalog of available implementations or their 313 features. Readers are advised to note that other implementations may 314 exist. 316 According to RFC 7942, "this will allow reviewers and working groups 317 to assign due consideration to documents that have the benefit of 318 running code, which may serve as evidence of valuable experimentation 319 and feedback that have made the implemented protocols more mature. 320 It is up to the individual working groups to use this information as 321 they see fit". 323 Organization: Stripe 325 o Description: Stripe uses custom HTTP header named "Idempotency- 326 Key" 328 o Reference: https://stripe.com/docs/idempotency 330 Organization: Adyen 332 o Description: Adyen uses custom HTTP header named "Idempotency-Key" 334 o Reference: https://docs.adyen.com/development-resources/api- 335 idempotency/ 337 Organization: Dwolla 339 o Description: Dwolla uses custom HTTP header named "Idempotency- 340 Key" 342 o Reference: https://docs.dwolla.com/ 344 Organization: Interledger 346 o Description: Interledger uses custom HTTP header named 347 "Idempotency-Key" 349 o Reference: https://github.com/interledger/ 351 Organization: WorldPay 353 o Description: WorldPay uses custom HTTP header named "Idempotency- 354 Key" 356 o Reference: https://developer.worldpay.com/docs/wpg/idempotency 358 Organization: Yandex 360 o Description: Yandex uses custom HTTP header named "Idempotency- 361 Key" 363 o Reference: https://cloud.yandex.com/docs/api-design- 364 guide/concepts/idempotency 366 4.1. Implementing the Concept 368 This is a list of implementations that implement the general concept, 369 but do so using different mechanisms: 371 Organization: Django 373 o Description: Django uses custom HTTP header named 374 "HTTP_IDEMPOTENCY_KEY" 376 o Reference: https://pypi.org/project/django-idempotency-key 378 Organization: Twilio 380 o Description: Twilio uses custom HTTP header named "I-Twilio- 381 Idempotency-Token" in webhooks 383 o Reference: https://www.twilio.com/docs/usage/webhooks/webhooks- 384 connection-overrides 386 Organization: PayPal 388 o Description: PayPal uses custom HTTP header named "PayPal-Request- 389 Id" 391 o Reference: https://developer.paypal.com/docs/business/develop/ 392 idempotency 394 Organization: RazorPay 396 o Description: RazorPay uses custom HTTP header named "X-Payout- 397 Idempotency" 399 o Reference: https://razorpay.com/docs/razorpayx/api/idempotency/ 401 Organization: OpenBanking 403 o Description: OpenBanking uses custom HTTP header called "x- 404 idempotency-key" 406 o Reference: https://openbankinguk.github.io/read-write-api- 407 site3/v3.1.6/profiles/read-write-data-api-profile.html#request- 408 headers 410 Organization: Square 412 o Description: To make an idempotent API call, Square recommends 413 adding a property named "idempotency_key" with a unique value in 414 the request body. 416 o Reference: https://developer.squareup.com/docs/build-basics/using- 417 rest-api 419 Organization: Google Standard Payments 421 o Description: Google Standard Payments API uses a property named 422 "requestId" in request body in order to provider idempotency in 423 various use cases. 425 o Reference: https://developers.google.com/standard-payments/ 426 payment-processor-service-api/rest/v1/TopLevel/capture 428 Organization: BBVA 430 o Description: BBVA Open Platform uses custom HTTP header called "X- 431 Unique-Transaction-ID" 433 o Reference: 434 https://bbvaopenplatform.com/apiReference/APIbasics/content/x- 435 unique-transaction-id 437 Organization: WebEngage 439 o Description: WebEngage uses custom HTTP header called "x-request- 440 id" to identify webhook POST requests uniquely to achieve events 441 idempotency. 443 o Reference: https://docs.webengage.com/docs/webhooks 445 5. Security Considerations 447 This section is meant to inform developers, information providers, 448 and users of known security concerns specific to the idempotency 449 keys. 451 For idempotent request handling, the resources MAY make use of the 452 value in the idempotency key to look up a cache or a persistent store 453 for duplicate requests matching the key. If the resource does not 454 validate the value of the idempotency key prior to performing such a 455 lookup, it MAY lead to various forms of security attacks and 456 compromise. To avoid such situations, the resource SHOULD publish 457 the expected format of the idempotency key, algorithm used to 458 generate it and always validate the key value as per the published 459 specification before processing any request. 461 6. Examples 463 The first example shows an idempotency-key header field with key 464 value using UUID version 4 scheme: 466 Idempotency-Key: "8e03978e-40d5-43e8-bc93-6894a57f9324" 468 Second example shows an idempotency-key header field with key value 469 using some random string generator: 471 Idempotency-Key: "clkyoesmbgybucifusbbtdsbohtyuuwz" 473 7. References 475 7.1. Normative References 477 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 478 Requirement Levels", BCP 14, RFC 2119, 479 DOI 10.17487/RFC2119, March 1997, 480 . 482 [RFC3864] Klyne, G., Nottingham, M., and J. Mogul, "Registration 483 Procedures for Message Header Fields", BCP 90, RFC 3864, 484 DOI 10.17487/RFC3864, September 2004, 485 . 487 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 488 Unique IDentifier (UUID) URN Namespace", RFC 4122, 489 DOI 10.17487/RFC4122, July 2005, 490 . 492 [RFC4918] Dusseault, L., Ed., "HTTP Extensions for Web Distributed 493 Authoring and Versioning (WebDAV)", RFC 4918, 494 DOI 10.17487/RFC4918, June 2007, 495 . 497 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 498 Specifications: ABNF", STD 68, RFC 5234, 499 DOI 10.17487/RFC5234, January 2008, 500 . 502 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 503 Protocol (HTTP/1.1): Message Syntax and Routing", 504 RFC 7230, DOI 10.17487/RFC7230, June 2014, 505 . 507 [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 508 Protocol (HTTP/1.1): Semantics and Content", RFC 7231, 509 DOI 10.17487/RFC7231, June 2014, 510 . 512 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 513 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 514 May 2017, . 516 7.2. Informative References 518 [RFC7942] Sheffer, Y. and A. Farrel, "Improving Awareness of Running 519 Code: The Implementation Status Section", BCP 205, 520 RFC 7942, DOI 10.17487/RFC7942, July 2016, 521 . 523 7.3. URIs 525 [1] https://github.com/paypal/api-standards/blob/master/ 526 patterns.md#idempotency 528 [2] https://stripe.com/docs/idempotency 530 [3] https://tools.ietf.org/html/draft-nottingham-http-poe-00 532 Appendix A. Acknowledgments 534 The authors would like to thank Mark Nottingham for his support for 535 this Internet Draft. We would like to acknowledge that this draft is 536 inspired by Idempotency related patterns described in API 537 documentation of PayPal [1] and Stripe [2] as well as Internet Draft 538 on POST Once Exactly [3] authored by Mark Nottingham. 540 The authors take all responsibility for errors and omissions. 542 Appendix B. Appendix 544 B.1. Appendix A. Imported ABNF 546 The following core rules are included by reference, as defined in 547 Appendix B.1 of [RFC5234]: ALPHA (letters), CR (carriage return), 548 CRLF (CR LF), CTL (controls), DIGIT (decimal 0-9), DQUOTE (double 549 quote), HEXDIG (hexadecimal 0-9/A-F/a-f), LF (line feed), OCTET (any 550 8-bit sequence of data), SP (space), and VCHAR (any visible US-ASCII 551 character). 553 The rules below are defined in [RFC7230]: 555 obs-text = 557 Authors' Addresses 559 Jayadeba Jena 560 PayPal, Inc. 562 Email: jjena@paypal.com 563 Sanjay Dalal 565 Email: sanjay.dalal@cal.berkeley.edu 566 URI: https://github.com/sdatspun2