idnits 2.17.1 draft-ietf-httpapi-idempotency-key-header-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 589 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (1 July 2021) is 1028 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7231 (Obsoleted by RFC 9110) Summary: 2 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Jena 3 Internet-Draft PayPal, Inc. 4 Intended status: Standards Track S. Dalal 5 Expires: 2 January 2022 1 July 2021 7 The Idempotency-Key HTTP Header Field 8 draft-ietf-httpapi-idempotency-key-header-00 10 Abstract 12 The HTTP Idempotency-Key request header field can be used to carry 13 idempotency key in order to make non-idempotent HTTP methods such as 14 "POST" or "PATCH" fault-tolerant. 16 Status of This Memo 18 This Internet-Draft is submitted in full conformance with the 19 provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF). Note that other groups may also distribute 23 working documents as Internet-Drafts. The list of current Internet- 24 Drafts is at https://datatracker.ietf.org/drafts/current/. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet-Drafts as reference 29 material or to cite them other than as "work in progress." 31 This Internet-Draft will expire on 2 January 2022. 33 Copyright Notice 35 Copyright (c) 2021 IETF Trust and the persons identified as the 36 document authors. All rights reserved. 38 This document is subject to BCP 78 and the IETF Trust's Legal 39 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 40 license-info) in effect on the date of publication of this document. 41 Please review these documents carefully, as they describe your rights 42 and restrictions with respect to this document. Code Components 43 extracted from this document must include Simplified BSD License text 44 as described in Section 4.e of the Trust Legal Provisions and are 45 provided without warranty as described in the Simplified BSD License. 47 Table of Contents 49 1. Introduction 50 1.1. Notational Conventions 51 2. The Idempotency-Key HTTP Request Header Field 52 2.1. Syntax 53 2.2. Uniqueness of Idempotency Key 54 2.3. Idempotency Key Validity and Expiry 55 2.4. Idempotency Fingerprint 56 2.5. Responsibilities 57 2.6. Idempotency Enforcement Scenarios 58 2.7. Error Scenarios 59 3. IANA Considerations 60 3.1. The Idempotency-Key HTTP Request Header Field 61 4. Implementation Status 62 4.1. Implementing the Concept 63 5. Security Considerations 64 6. Examples 65 7. References 66 7.1. Normative References 67 7.2. Informative References 68 Appendix A. Acknowledgments 69 Appendix B. Appendix 70 B.1. Appendix A. Imported ABNF 71 Authors' Addresses 73 1. Introduction 75 Idempotence is the property of certain operations in mathematics and 76 computer science whereby they can be applied multiple times without 77 changing the result beyond the initial application. It does not 78 matter if the operation is called only once, or 10s of times over. 79 The result SHOULD be the same. 81 Idempotency is important in building a fault-tolerant HTTP API. An 82 HTTP request method is considered "idempotent" if the intended effect 83 on the server of multiple identical requests with that method is the 84 same as the effect for a single such request. According to 85 [RFC7231], HTTP methods "OPTIONS", "HEAD", "GET", "PUT" and "DELETE" 86 are idempotent while methods "POST" and "PATCH" are not. 88 Let's say a client of an HTTP API wants to create (or update) a 89 resource using a "POST" method. Since "POST" is NOT an idempotent 90 method, calling it multiple times can result in duplication or wrong 91 updates. Consider a scenario where the client sent a "POST" request 92 to the server, but it got a timeout. Following questions arise : Is 93 the resource actually created (or updated)? Did the timeout occur 94 during sending of the request, or when receiving of the response? 95 Can the client safely retry the request, or does it need to figure 96 out what happened in the first place? If "POST" had been an 97 idempotent method, such questions may not arise. Client would safely 98 retry a request until it actually gets a valid response from the 99 server. 101 For many use cases of HTTP API, duplicate resource is a severe 102 problem from business perspective. For example, duplicate records 103 for requests involving any kind of money transfer "MUST NOT" be 104 allowed. In other cases, processing of duplicate webhook delivery is 105 not expected. 107 1.1. Notational Conventions 109 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 110 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 111 "OPTIONAL" in this document are to be interpreted as described in BCP 112 14 [RFC2119] [RFC8174] when, and only when, they appear in all 113 capitals, as shown here, and without quotes. 115 This specification uses the Augmented Backus-Naur Form (ABNF) 116 notation of [RFC5234] and includes, by reference, the IMF-fixdate 117 rule as defined in Section 7.1.1.1 of [RFC7231]. 119 The term "resource" is to be interpreted as defined in Section 2 of 120 [RFC7231], that is identified by an URI. The term "resource server" 121 is to be interpreted as "origin server" as defined in Section 3 of 122 [RFC7231]. 124 2. The Idempotency-Key HTTP Request Header Field 126 An idempotency key is a unique value generated by the client which 127 the resource server uses to recognize subsequent retries of the same 128 request. The "Idempotency-Key" HTTP request header field carries 129 this key. 131 2.1. Syntax 133 The "Idempotency-Key" request header field describes 135 Idempotency-Key = idempotency-key-value 137 idempotency-key-value = opaque-value 138 opaque-value = DQUOTE *idempotencyvalue DQUOTE 139 idempotencyvalue = %x21 / %x23-7E / obs-text 140 ; VCHAR except double quotes, plus obs-text 142 Clients MUST NOT include more than one "Idempotency-Key" header field 143 in the same request. 145 The following example shows an idempotency key using "UUID" 146 [RFC4122]: 148 Idempotency-Key: "8e03978e-40d5-43e8-bc93-6894a57f9324" 150 2.2. Uniqueness of Idempotency Key 152 The idempotency key that is supplied as part of every "POST" request 153 MUST be unique and MUST NOT be reused with another request with a 154 different request payload. 156 Uniqueness of the key MUST be defined by the resource owner and MUST 157 be implemented by the clients of the resource server. It is 158 RECOMMENDED that "UUID" [RFC4122] or a similar random identifier be 159 used as an idempotency key. 161 2.3. Idempotency Key Validity and Expiry 163 The resource MAY enforce time based idempotency keys, thus, be able 164 to purge or delete a key upon its expiry. The resource server SHOULD 165 define such expiration policy and publish in related documentation. 167 2.4. Idempotency Fingerprint 169 An idempotency fingerprint MAY be used in conjunction with an 170 idempotency key to determine the uniqueness of a request. Such a 171 fingerprint is generated from request payload data by the resource 172 server. An idempotency fingerprint generation algorithm MAY use one 173 of the following or similar approaches to create a fingerprint. 175 * Checksum of the entire request payload. 177 * Checksum of selected element(s) in the request payload. 179 * Field value match for each field in the request payload. 181 * Field value match for selected element(s) in the request payload. 183 * Request digest/signature. 185 2.5. Responsibilities 187 Client 189 Clients of HTTP API requiring idempotency, SHOULD understand the 190 idempotency related requirements as published by the server and use 191 appropriate algorithm to generate idempotency keys. 193 For each request, client SHOULD 195 * Send a unique idempotency key in the HTTP "Idempotency-Key" 196 request header field. 198 Resource Server 200 Resource server MUST publish idempotency related specification. This 201 specification MUST include expiration related policy if applicable. 202 Server is responsible for managing the lifecycle of the idempotency 203 key. 205 For each request, server SHOULD 207 * Identify idempotency key from the HTTP "Idempotency-Key" request 208 header field. 210 * Generate idempotency fingerprint if required. 212 * Check for idempotency considering various scenarios including the 213 ones described in section below. 215 2.6. Idempotency Enforcement Scenarios 217 * First time request (idempotency key or fingerprint has not been 218 seen) 220 The resource server SHOULD process the request normally and 221 respond with an appropriate response and status code. 223 * Duplicate request (idempotency key or fingerprint has been seen) 225 Retry 227 The request was retried after the original request completed. The 228 resource server MUST respond with the result of the previously 229 completed operation, success or an error. 231 Concurrent Request 233 The request was retried before the original request completed. 234 The resource server MUST respond with a resource conflict error. 235 See Error Scenarios for details. 237 2.7. Error Scenarios 239 If the "Idempotency-Key" request header is missing for a documented 240 idempotent operation requiring this header, the resource server MUST 241 reply with an HTTP "400" status code with body containing a link 242 pointing to relevant documentation. Alternately, using the HTTP 243 header "Link", the client can be informed about the error as shown 244 below. 246 HTTP/1.1 400 Bad Request 247 Link: ; 248 rel="describedby"; type="text/html" 250 If there is an attempt to reuse an idempotency key with a different 251 request payload, the resource server MUST reply with a HTTP "422" 252 status code with body containing a link pointing to relevant 253 documentation. The status code "422" is defined in Section 11.2 of 254 [RFC4918]. The server can also inform the client by using the HTTP 255 header "Link" as shown below. 257 HTTP/1.1 422 Unprocessable Entity 258 Link: ; 259 rel="describedby"; type="text/html" 261 If the request is retried, while the original request is still being 262 processed, the resource server MUST reply with an HTTP "409" status 263 code with body containing a link or the HTTP header "Link" pointing 264 to the relevant documentation. 266 HTTP/1.1 409 Conflict 267 Link: ; 268 rel="describedby"; type="text/html" 270 For other errors, the resource MUST return the appropriate status 271 code and error message. 273 3. IANA Considerations 275 3.1. The Idempotency-Key HTTP Request Header Field 277 The "Idempotency-Key" request header should be added to the permanent 278 registry of message header fields (see [RFC3864]), taking into 279 account the guidelines given by HTTP/1.1 [RFC7231]. 281 Header Field Name: Idempotency-Key 283 Applicable Protocol: Hypertext Transfer Protocol (HTTP) 285 Status: Standard 287 Authors: 288 Jayadeba Jena 289 Email: jjena@paypal.com 291 Sanjay Dalal 292 Email: sanjay.dalal@cal.berkeley.edu 294 Change controller: IETF 296 Specification document: this specification, 297 Section 2 "The Idempotency-Key HTTP Request Header Field" 299 4. Implementation Status 301 Note to RFC Editor: Please remove this section before publication. 303 This section records the status of known implementations of the 304 protocol defined by this specification at the time of posting of this 305 Internet-Draft, and is based on a proposal described in [RFC7942]. 306 The description of implementations in this section is intended to 307 assist the IETF in its decision processes in progressing drafts to 308 RFCs. Please note that the listing of any individual implementation 309 here does not imply endorsement by the IETF. Furthermore, no effort 310 has been spent to verify the information presented here that was 311 supplied by IETF contributors. This is not intended as, and must not 312 be construed to be, a catalog of available implementations or their 313 features. Readers are advised to note that other implementations may 314 exist. 316 According to RFC 7942, "this will allow reviewers and working groups 317 to assign due consideration to documents that have the benefit of 318 running code, which may serve as evidence of valuable experimentation 319 and feedback that have made the implemented protocols more mature. 320 It is up to the individual working groups to use this information as 321 they see fit". 323 Organization: Stripe 325 * Description: Stripe uses custom HTTP header named "Idempotency- 326 Key" 328 * Reference: https://stripe.com/docs/idempotency 330 Organization: Adyen 332 * Description: Adyen uses custom HTTP header named "Idempotency-Key" 334 * Reference: https://docs.adyen.com/development-resources/api- 335 idempotency/ 337 Organization: Dwolla 339 * Description: Dwolla uses custom HTTP header named "Idempotency- 340 Key" 342 * Reference: https://docs.dwolla.com/ 344 Organization: Interledger 346 * Description: Interledger uses custom HTTP header named 347 "Idempotency-Key" 349 * Reference: https://github.com/interledger/ 351 Organization: WorldPay 353 * Description: WorldPay uses custom HTTP header named "Idempotency- 354 Key" 356 * Reference: https://developer.worldpay.com/docs/wpg/idempotency 358 Organization: Yandex 360 * Description: Yandex uses custom HTTP header named "Idempotency- 361 Key" 363 * Reference: https://cloud.yandex.com/docs/api-design- 364 guide/concepts/idempotency 366 Organization: http4s.org 368 * Description: Http4s is a minimal, idiomatic Scala interface for 369 HTTP services. 371 * Reference: https://github.com/http4s/http4s 373 Organization: Finastra 375 * Description: Finastra uses custom HTTP header named "Idempotency- 376 Key" 378 * Reference: https://developer.fusionfabric.cloud/ 380 Organization: Datatrans 382 * Description: Datatrans focuses on the technical processing of 383 payments, including hosting smart payment forms and correctly 384 routing payment information. 386 * Reference: https://docs.datatrans.ch/docs/api-endpoints 388 4.1. Implementing the Concept 390 This is a list of implementations that implement the general concept, 391 but do so using different mechanisms: 393 Organization: Django 395 * Description: Django uses custom HTTP header named 396 "HTTP_IDEMPOTENCY_KEY" 398 * Reference: https://pypi.org/project/django-idempotency-key 400 Organization: Twilio 402 * Description: Twilio uses custom HTTP header named "I-Twilio- 403 Idempotency-Token" in webhooks 405 * Reference: https://www.twilio.com/docs/usage/webhooks/webhooks- 406 connection-overrides 408 Organization: PayPal 410 * Description: PayPal uses custom HTTP header named "PayPal-Request- 411 Id" 413 * Reference: https://developer.paypal.com/docs/business/develop/ 414 idempotency 416 Organization: RazorPay 418 * Description: RazorPay uses custom HTTP header named "X-Payout- 419 Idempotency" 421 * Reference: https://razorpay.com/docs/razorpayx/api/idempotency/ 423 Organization: OpenBanking 425 * Description: OpenBanking uses custom HTTP header called "x- 426 idempotency-key" 428 * Reference: https://openbankinguk.github.io/read-write-api- 429 site3/v3.1.6/profiles/read-write-data-api-profile.html#request- 430 headers 432 Organization: Square 434 * Description: To make an idempotent API call, Square recommends 435 adding a property named "idempotency_key" with a unique value in 436 the request body. 438 * Reference: https://developer.squareup.com/docs/build-basics/using- 439 rest-api 441 Organization: Google Standard Payments 443 * Description: Google Standard Payments API uses a property named 444 "requestId" in request body in order to provider idempotency in 445 various use cases. 447 * Reference: https://developers.google.com/standard-payments/ 448 payment-processor-service-api/rest/v1/TopLevel/capture 450 Organization: BBVA 452 * Description: BBVA Open Platform uses custom HTTP header called "X- 453 Unique-Transaction-ID" 455 * Reference: 456 https://bbvaopenplatform.com/apiReference/APIbasics/content/x- 457 unique-transaction-id 459 Organization: WebEngage 461 * Description: WebEngage uses custom HTTP header called "x-request- 462 id" to identify webhook POST requests uniquely to achieve events 463 idempotency. 465 * Reference: https://docs.webengage.com/docs/webhooks 467 5. Security Considerations 469 This section is meant to inform developers, information providers, 470 and users of known security concerns specific to the idempotency 471 keys. 473 For idempotent request handling, the resources MAY make use of the 474 value in the idempotency key to look up a cache or a persistent store 475 for duplicate requests matching the key. If the resource does not 476 validate the value of the idempotency key prior to performing such a 477 lookup, it MAY lead to various forms of security attacks and 478 compromise. To avoid such situations, the resource SHOULD publish 479 the expected format of the idempotency key, algorithm used to 480 generate it and always validate the key value as per the published 481 specification before processing any request. 483 6. Examples 485 The first example shows an idempotency-key header field with key 486 value using UUID version 4 scheme: 488 Idempotency-Key: "8e03978e-40d5-43e8-bc93-6894a57f9324" 490 Second example shows an idempotency-key header field with key value 491 using some random string generator: 493 Idempotency-Key: "clkyoesmbgybucifusbbtdsbohtyuuwz" 495 7. References 497 7.1. Normative References 499 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 500 Requirement Levels", BCP 14, RFC 2119, 501 DOI 10.17487/RFC2119, March 1997, 502 . 504 [RFC3864] Klyne, G., Nottingham, M., and J. Mogul, "Registration 505 Procedures for Message Header Fields", BCP 90, RFC 3864, 506 DOI 10.17487/RFC3864, September 2004, 507 . 509 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 510 Unique IDentifier (UUID) URN Namespace", RFC 4122, 511 DOI 10.17487/RFC4122, July 2005, 512 . 514 [RFC4918] Dusseault, L., Ed., "HTTP Extensions for Web Distributed 515 Authoring and Versioning (WebDAV)", RFC 4918, 516 DOI 10.17487/RFC4918, June 2007, 517 . 519 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 520 Specifications: ABNF", STD 68, RFC 5234, 521 DOI 10.17487/RFC5234, January 2008, 522 . 524 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 525 Protocol (HTTP/1.1): Message Syntax and Routing", 526 RFC 7230, DOI 10.17487/RFC7230, June 2014, 527 . 529 [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 530 Protocol (HTTP/1.1): Semantics and Content", RFC 7231, 531 DOI 10.17487/RFC7231, June 2014, 532 . 534 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 535 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 536 May 2017, . 538 7.2. Informative References 540 [RFC7942] Sheffer, Y. and A. Farrel, "Improving Awareness of Running 541 Code: The Implementation Status Section", BCP 205, 542 RFC 7942, DOI 10.17487/RFC7942, July 2016, 543 . 545 Appendix A. Acknowledgments 547 The authors would like to thank Mark Nottingham for his support for 548 this Internet Draft. We would like to acknowledge that this draft is 549 inspired by Idempotency related patterns described in API 550 documentation of PayPal (https://github.com/paypal/api- 551 standards/blob/master/patterns.md#idempotency) and Stripe 552 (https://stripe.com/docs/idempotency) as well as Internet Draft on 553 POST Once Exactly (https://tools.ietf.org/html/draft-nottingham-http- 554 poe-00) authored by Mark Nottingham. 556 The authors take all responsibility for errors and omissions. 558 Appendix B. Appendix 560 B.1. Appendix A. Imported ABNF 562 The following core rules are included by reference, as defined in 563 Appendix B.1 of [RFC5234]: ALPHA (letters), CR (carriage return), 564 CRLF (CR LF), CTL (controls), DIGIT (decimal 0-9), DQUOTE (double 565 quote), HEXDIG (hexadecimal 0-9/A-F/a-f), LF (line feed), OCTET (any 566 8-bit sequence of data), SP (space), and VCHAR (any visible US-ASCII 567 character). 569 The rules below are defined in [RFC7230]: 571 obs-text = 573 Authors' Addresses 575 Jayadeba Jena 576 PayPal, Inc. 578 Email: jjena@paypal.com 580 Sanjay Dalal 582 Email: sanjay.dalal@cal.berkeley.edu 583 URI: https://github.com/sdatspun2