---
title: "Idempotency Keys for Payment APIs: Byte-for-Byte Replays | CloudGuys Blog"
description: "How idempotency keys make payment retries safe: exactly-once execution in Postgres, byte-identical replays, lost commits across services, and a comparison."
url: "https://cloudguys.io/blog/idempotency-keys-money-apis"
language: "en"
image: "https://cloudguys.io/_astro/cover.D9PepFpj.jpg"
published: "2026-11-18T00:00:00.000Z"
modified: "2026-11-18T00:00:00.000Z"
---

ENGINEERING NOTES / FINTECH ENGINEERING

# Idempotency Keys for Payment APIs: What "Replay Byte for Byte" Means

An idempotency key makes a timed-out payment safe to retry. What exactly-once really requires in the database, why the replay should be byte-identical, what happens across two services, and how Stripe, Adyen and the IETF draft differ.

Sabbir Ahmed · · · November 18, 2026 · · · 9 min read · · · Facts checked November 8, 2026 · · · About WalletD

![Idempotency Keys for Payment APIs: What "Replay Byte for Byte" Means](https://cloudguys.io/_astro/cover.D9PepFpj_1V13WG.webp)

An idempotency key lets a client retry a money-moving request without moving the money twice: the first request with a given key runs, and every later request with the same key gets the first response back instead of running again. “Replay byte for byte” means that returned response is the identical status code and identical body bytes, not a freshly rendered copy, so a client that compares, hashes or logs responses sees exactly one outcome.

The header is the easy part. What takes work is making “exactly once” true when two duplicates arrive at the same instant, when the server crashes between steps, and when the money lives in a different service from the request. Below is how the WalletD API handles each case, with the code and tests behind every claim, and how that compares with the published behaviour of Stripe, Adyen and the IETF draft. CloudGuys builds WalletD. Facts were checked on 8 November 2026.

## The short answer

-   **Require the key on every money-moving `POST`.** Reads and configuration changes do not need one.
-   **Store the key in the same database transaction as the money.** A key that commits separately from the work it guards is a race.
-   **Hash the request and refuse a reused key with a different body** (WalletD answers `409 idempotency_key_reuse`).
-   **Return the stored bytes on replay,** including to a duplicate that arrived while the first was still running.
-   **Derive keys from your business object,** such as `payment-order-4471`, never a fresh UUID per attempt.
-   **Across services, derive downstream keys from the upstream one,** and let the caller ask the downstream service what already happened.

## What does an idempotency key actually guarantee?

One logical operation has at most one effect, however many times the request is sent. A network can fail after the server acted: the transfer happened, the response was lost, and the client cannot tell. With a key, the client resends with the same key and gets either the stored result or, if nothing committed, a first execution.

That only works if the client keeps the key stable. WalletD’s developer docs name the failure modes: a `uuid4()` generated per attempt, or a timestamp key, means “every retry moves money”, and a constant key like `transfer` means your second, unrelated transfer is refused as a replay. Good keys come from the business object: `payout-invoice-8891`, `transfer-{order_id}`. Keys are scoped per tenant, and merchant-bound API keys add no merchant namespace, so a marketplace should put the merchant in the key (`merchant-42-transfer-order-8891`).

## How do you make it exactly-once in the database?

Insert the key row first, inside the same transaction that does the work, and let the unique index arbitrate. WalletD’s arbiter (`internal/idempotency/idempotency.go`, recorded as ADR-0006) does four things:

1.  `INSERT INTO idempotency_keys (tenant_id, key, request_hash) ... ON CONFLICT DO NOTHING`, then run the operation in **the same transaction**, then store the response code and body on the key row.
2.  A concurrent duplicate’s insert **blocks on the winner’s uncommitted row** in PostgreSQL. If the winner commits, the duplicate’s insert affects zero rows and it reads the stored response. If the winner rolls back, the duplicate’s insert proceeds and it becomes the winner. The arbiter tries this up to three times.
3.  If the stored `request_hash` (SHA-256 of the request) differs from the new one, it refuses with `money.ErrIdempotencyReuse`, which the API maps to `409 idempotency_key_reuse`.
4.  A failed operation rolls back the key row too, so a retry after a refusal can succeed.

The core of it, trimmed from the source:

```go
tag, err := tx.Exec(ctx,
    `INSERT INTO idempotency_keys (tenant_id, key, request_hash)
     VALUES ($1, $2, $3) ON CONFLICT DO NOTHING`,
    tenantID, key, requestHash)
if tag.RowsAffected() == 0 {
    return errLostRace // someone else holds the key: read their stored result
}
result, err = fn(tx) // the money movement, same transaction
// RETURNING hands back the jsonb-normalized bytes so the winner's
// response is byte-identical to every future replay.
return tx.QueryRow(ctx,
    `UPDATE idempotency_keys SET response_code = $1, response_body = $2, txn_id = $3
     WHERE tenant_id = $4 AND key = $5 RETURNING response_body`,
    result.StatusCode, body, result.TxnID, tenantID, key,
).Scan(&result.Body)
```

Point 2 is the part to copy first: duplicates **wait** for the winner instead of getting an error, so the client never has to resolve a race.

## Why should the replay be byte-identical?

Because “the same result” is ambiguous, and clients depend on the bytes. If the server re-renders a replay from current state, the body can change: a status moved on, a field was added, numbers were reformatted. A client that logs the first response, or a webhook consumer that deduplicates on a body hash, then sees two different outcomes for one operation.

WalletD stores the response in a PostgreSQL `jsonb` column, and the winner returns the bytes PostgreSQL hands back through `RETURNING` rather than the bytes it rendered. `jsonb` normalises whitespace and key order, so the very first response and every replay are the same bytes. A test asserts it: `TestIdempotentReplayIsByteIdentical`.

## What happens when the money lives in another service?

You can no longer put the key and the money in one transaction, so you derive every downstream key from the upstream one and let the caller ask what already happened.

WalletD’s books live in a separate ledger service, ledgerd, with its own database. No transaction spans the two. The failure to design for is: ledgerd commits the posting, then walletd crashes before it records the payment row and the stored response. The retry must find that posting, not make a second one.

Three mechanisms close that window:

-   **Deterministic ids.** The payment id is a UUIDv5 of the tenant and the client’s key, and the ledger posting key derives from it (`pay:<payment id>:capture`, `refund:<refund id>`). A retry rebuilds the same posting request, and ledgerd’s own idempotency layer replays it.
-   **A stored-result probe.** ledgerd exposes `GET /v1/ledgers/{id}/idempotency/{key}`, which returns the stored result with no request-hash check, because the repairing caller may not be able to rebuild the original request (the fee schedule may have changed in between). A miss is authoritative inside the retention window: the posting never committed.
-   **Repair from the stored legs.** When a retry’s rebuilt request no longer matches, walletd recovers the committed posting and finishes its own records from *those* amounts, because the stored legs are the money that moved (ADR-0022). That ADR records eight money paths where the first version of this recovery was broken, all the same way: the retry re-checked mutable state the money had already left and refused before reaching the replay.

The practical consequence for an integrator, stated in the docs: a `5xx` does not prove that no money moved. Keep the original request and key, retry within a bounded budget, and never create a replacement payment with a new key.

## How long should keys live?

Long enough to cover every realistic retry, with the policy published. The IETF draft says a resource “SHOULD define such expiration policy and publish it in the documentation” ([draft-ietf-httpapi-idempotency-key-header-08](https://datatracker.ietf.org/doc/draft-ietf-httpapi-idempotency-key-header/), 22 October 2026).

In WalletD the two layers differ, and the docs say so. The public API’s idempotency records currently have no automatic expiry or cleanup. ledgerd keeps completed keys for a configurable retention, 30 days by default, purged by a daily job; after the purge a reused key would execute again. Retries arrive in seconds or minutes, so the repair window sits well inside it.

## How do Stripe, Adyen and the IETF draft compare?

Their published behaviour, read on 8 November 2026. We did not test the third-party APIs for this post; every cell outside the WalletD column comes from the linked documentation.

|  | WalletD API | Stripe (API v1) | Adyen | IETF draft-08 |
| --- | --- | --- | --- | --- |
| Header | `Idempotency-Key`, required on money-moving `POST` | `Idempotency-Key` | `idempotency-key` | `Idempotency-Key` |
| Key limits | Scoped per tenant | Up to 255 characters | Up to 64 characters, unique per company account | Defined by the resource |
| Same key, different body | `409 idempotency_key_reuse` | Error: parameters are compared with the original | Not stated on the page we read | `422` recommended |
| Duplicate while first is running | Waits, then gets the stored response | Not saved; you can retry | `422` or `409`, error code 704 | `409` recommended |
| What a replay returns | Stored status and body, byte-identical | Saved status and body, including `500`s | The first response | Defined by the resource |
| Retention | No automatic expiry at the API; ledger keys 30 days | Keys can be pruned after at least 24 hours | 7 to 14 days | Resource must publish its policy |

Sources: [Stripe idempotent requests](https://docs.stripe.com/api/idempotent_requests), [Adyen API idempotency](https://docs.adyen.com/development-resources/api-idempotency), [IETF draft-08](https://datatracker.ietf.org/doc/draft-ietf-httpapi-idempotency-key-header/). The draft is still an Internet-Draft, a work in progress, not an RFC.

Two differences matter in practice. WalletD returns `409` for a mismatched body where the draft recommends `422`, so if you build to the draft, handle both. And Stripe and Adyen document a concurrent duplicate as something the client deals with (retry, or a 409/422 error), while WalletD makes the duplicate wait for the winner. Neither approach is wrong, but your retry loop has to know which one it is talking to.

## What did we verify ourselves?

We ran the relevant WalletD suites on 8 November 2026 against disposable PostgreSQL containers and a real ledgerd started by the test harness (walletd at `f446353` with uncommitted working-tree changes in unrelated catalog files; ledgerd at `acf0cba`). All passed, among them:

```text
--- PASS: TestIdempotentReplayIsByteIdentical (8.06s)
--- PASS: TestIdempotentRejectsDifferentPayload (0.05s)
--- PASS: TestIdempotentReleasesKeyOnFailure (0.04s)
--- PASS: TestIdempotentConcurrentDuplicatesRunOnce (0.05s)
--- PASS: TestLedgerdInstantPaymentWithFeeAndCommission (7.97s)
--- PASS: TestLedgerdVoidRefusesAndCaptureRepairsAfterLostCaptureCommit (0.06s)
```

`TestIdempotentConcurrentDuplicatesRunOnce` sends 20 concurrent top-ups of 7,777 under one key and requires one transaction, a balance of exactly 7,777, and identical bodies for all 20 callers. The lost-commit test posts the ledger half by hand under the keys walletd derives, then runs the retry and asserts a single debit. In the [ledger pillar post](https://cloudguys.io/blog/double-entry-ledger-design-wallet-invariants) we describe the chaos gate, which adds 50 keys hit by four concurrent callers each inside 1,000 mixed operations.

## A checklist for your own money API

1.  Make the key mandatory on every money-moving endpoint and reject requests without it before doing anything.
2.  Insert the key row inside the business transaction; never check-then-insert in two steps.
3.  Store a request hash and refuse reuse with a different body, with a stable error code.
4.  Store and return the response bytes; do not re-render replays.
5.  Release the key when the operation fails, so a corrected retry can run.
6.  Derive every downstream key and id deterministically from the client’s key, and give callers a way to ask what committed.
7.  Publish the retention period and the status codes. Clients write retry loops against your documentation, not your code.

## FAQ

**Should the idempotency key be a random UUID?** Only if you generate it once per business operation and reuse it on every retry. A UUID generated per attempt means every retry moves money. Derive the key from the business object instead, such as `transfer-{order_id}`.

**What happens if I reuse a key with a different body?** WalletD refuses it with `409 idempotency_key_reuse`, because the stored SHA-256 request hash no longer matches. The IETF draft recommends `422` for the same case.

**Does a 5xx mean no money moved?** No. The posting may have committed before the failure. Retry with the original request and key within a bounded budget, and never create a replacement payment under a new key.

**How long does WalletD keep keys?** The public API’s records have no automatic expiry today. ledgerd keeps completed keys for 30 days by default, after which a reused key would execute again.

This is engineering guidance, not legal or compliance advice. Rules on holding customer funds, authorization windows and record keeping differ by country and licence, so check them with your counsel and regulator.

## Try it on a sandbox

WalletD requires an idempotency key on every money-moving call and documents the retry rules in six languages. It is in early access, with no public pricing; the [WalletD product page](https://cloudguys.io/products/walletd) has the overview. To see the replay behaviour against a sandbox tenant, [request a WalletD walkthrough](https://walletd.io/demo?utm_source=cloudguys_blog&utm_medium=referral&utm_campaign=idempotency-keys-money-apis).

#Idempotency · #Payments API · #API Design · #PostgreSQL · #WalletD

Share this article

-   [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcloudguys.io%2Fblog%2Fidempotency-keys-money-apis%3Futm_source%3Dlinkedin%26utm_medium%3Dsocial "Share on LinkedIn")
-   [X](https://twitter.com/intent/tweet?text=Idempotency%20Keys%20for%20Payment%20APIs%3A%20What%20%22Replay%20Byte%20for%20Byte%22%20Means&url=https%3A%2F%2Fcloudguys.io%2Fblog%2Fidempotency-keys-money-apis%3Futm_source%3Dx%26utm_medium%3Dsocial "Share on X")
-   [Reddit](https://www.reddit.com/submit?url=https%3A%2F%2Fcloudguys.io%2Fblog%2Fidempotency-keys-money-apis%3Futm_source%3Dreddit%26utm_medium%3Dsocial&title=Idempotency%20Keys%20for%20Payment%20APIs%3A%20What%20%22Replay%20Byte%20for%20Byte%22%20Means "Share on Reddit")
-   [Hacker News](https://news.ycombinator.com/submitlink?u=https%3A%2F%2Fcloudguys.io%2Fblog%2Fidempotency-keys-money-apis%3Futm_source%3Dhackernews%26utm_medium%3Dsocial&t=Idempotency%20Keys%20for%20Payment%20APIs%3A%20What%20%22Replay%20Byte%20for%20Byte%22%20Means "Share on Hacker News")
-   [Email](mailto:?subject=Idempotency%20Keys%20for%20Payment%20APIs%3A%20What%20%22Replay%20Byte%20for%20Byte%22%20Means&body=Idempotency%20Keys%20for%20Payment%20APIs%3A%20What%20%22Replay%20Byte%20for%20Byte%22%20Means%0A%0Ahttps%3A%2F%2Fcloudguys.io%2Fblog%2Fidempotency-keys-money-apis%3Futm_source%3Demail%26utm_medium%3Dsocial "Share on Email")

BUILT AND OPERATED BY CLOUDGUYS

## WalletD: wallet infrastructure  
Wallet infrastructure for business, on a ledger you can audit.

Merchant collections, balances, payments, rewards and payouts behind one API, with every movement posted to a separate double-entry ledger that reconciles by construction. Run it on your own infrastructure or in the cloud.

[Visit WalletD ↗](https://walletd.io/?utm_source=cloudguys_blog&utm_medium=referral&utm_campaign=idempotency-keys-money-apis) [About WalletD ↗](https://cloudguys.io/products/walletd)

## Structured data

```json
[
  {
    "@context": "https://schema.org",
    "@type": [
      "ProfessionalService",
      "Organization"
    ],
    "@id": "https://cloudguys.io/#organization",
    "name": "CloudGuys",
    "url": "https://cloudguys.io/",
    "logo": {
      "@type": "ImageObject",
      "url": "https://cloudguys.io/assets/logo.png",
      "width": 512,
      "height": 512
    },
    "image": "https://cloudguys.io/assets/og-image.png",
    "description": "Self-hosted software products, senior engineers embedded in client teams, and cloud security and architecture advisory across AWS, Google Cloud, Azure, Kubernetes, and infrastructure as code.",
    "email": "hello@cloudguys.io",
    "founder": {
      "@type": "Person",
      "@id": "https://cloudguys.io/authors/sabbir-ahmed#person",
      "name": "Sabbir Ahmed",
      "url": "https://cloudguys.io/authors/sabbir-ahmed"
    },
    "slogan": "Ship with confidence. Keep control.",
    "areaServed": "Worldwide",
    "hasOfferCatalog": {
      "@type": "OfferCatalog",
      "name": "CloudGuys engagements",
      "itemListElement": [
        {
          "@type": "Offer",
          "itemOffered": {
            "@type": "Service",
            "name": "Embedded engineers",
            "url": "https://cloudguys.io/services/embedded-engineers",
            "provider": {
              "@id": "https://cloudguys.io/#organization"
            }
          }
        },
        {
          "@type": "Offer",
          "itemOffered": {
            "@type": "Service",
            "name": "Cloud security assessment",
            "url": "https://cloudguys.io/services/cloud-security-assessment",
            "provider": {
              "@id": "https://cloudguys.io/#organization"
            }
          }
        },
        {
          "@type": "Offer",
          "itemOffered": {
            "@type": "Service",
            "name": "Cloud architecture review",
            "url": "https://cloudguys.io/services/cloud-architecture-review",
            "provider": {
              "@id": "https://cloudguys.io/#organization"
            }
          }
        },
        {
          "@type": "Offer",
          "itemOffered": {
            "@type": "Service",
            "name": "Security remediation",
            "url": "https://cloudguys.io/services/security-remediation",
            "provider": {
              "@id": "https://cloudguys.io/#organization"
            }
          }
        }
      ]
    },
    "knowsAbout": [
      "DevOps",
      "Kubernetes",
      "Terraform",
      "Cloud migration",
      "Cloud cost optimization",
      "GitOps",
      "Site reliability engineering",
      "Kubernetes dashboard",
      "AI SRE",
      "Kubernetes troubleshooting",
      "AWS",
      "Google Cloud",
      "Azure"
    ],
    "contactPoint": {
      "@type": "ContactPoint",
      "contactType": "sales",
      "email": "hello@cloudguys.io",
      "availableLanguage": [
        "English"
      ],
      "areaServed": "Worldwide"
    },
    "sameAs": [
      "https://www.linkedin.com/company/agrohitech",
      "https://berth.cloudguys.io",
      "https://tele.cloudguys.io"
    ],
    "owns": [
      {
        "@type": "SoftwareApplication",
        "@id": "https://berth.cloudguys.io/#software",
        "name": "Berth",
        "applicationCategory": "DeveloperApplication",
        "operatingSystem": "Kubernetes",
        "url": "https://berth.cloudguys.io",
        "description": "Berth is a self-hosted Kubernetes dashboard with a read-only AI SRE: endpoint health, capacity planning, guided app exposure, and evidence-based troubleshooting, run in your own cluster."
      },
      {
        "@type": "SoftwareApplication",
        "@id": "https://tele.cloudguys.io/#software",
        "name": "Tele",
        "applicationCategory": "CommunicationApplication",
        "operatingSystem": "Web",
        "url": "https://tele.cloudguys.io",
        "description": "Tele is real-time audio and video call infrastructure: a meeting API for embedding calls in your own application and a managed workspace for teams, deployable in your own cloud or private environment."
      }
    ]
  },
  {
    "@context": "https://schema.org",
    "@type": "WebSite",
    "@id": "https://cloudguys.io/#website",
    "url": "https://cloudguys.io/",
    "name": "CloudGuys",
    "description": "Self-hosted software, embedded engineers and cloud security advisory for teams that run their own platform.",
    "publisher": {
      "@id": "https://cloudguys.io/#organization"
    },
    "inLanguage": "en"
  },
  {
    "@context": "https://schema.org",
    "@type": "BlogPosting",
    "@id": "https://cloudguys.io/blog/idempotency-keys-money-apis#article",
    "headline": "Idempotency Keys for Payment APIs: What \"Replay Byte for Byte\" Means",
    "description": "How idempotency keys make payment retries safe: exactly-once execution in Postgres, byte-identical replays, lost commits across services, and a comparison.",
    "image": "https://cloudguys.io/_astro/cover.D9PepFpj.jpg",
    "datePublished": "2026-11-18T00:00:00.000Z",
    "dateModified": "2026-11-18T00:00:00.000Z",
    "author": {
      "@type": "Person",
      "@id": "https://cloudguys.io/authors/sabbir-ahmed#person",
      "name": "Sabbir Ahmed",
      "jobTitle": "Founder",
      "url": "https://cloudguys.io/authors/sabbir-ahmed",
      "worksFor": {
        "@id": "https://cloudguys.io/#organization"
      },
      "sameAs": [
        "https://github.com/by-sabbir"
      ]
    },
    "publisher": {
      "@id": "https://cloudguys.io/#organization"
    },
    "mainEntityOfPage": {
      "@type": "WebPage",
      "@id": "https://cloudguys.io/blog/idempotency-keys-money-apis"
    },
    "articleSection": "Fintech Engineering",
    "keywords": "Idempotency, Payments API, API Design, PostgreSQL, WalletD",
    "inLanguage": "en",
    "about": {
      "@type": "SoftwareApplication",
      "@id": "https://walletd.io/#software",
      "name": "WalletD",
      "url": "https://walletd.io"
    }
  },
  {
    "@context": "https://schema.org",
    "@type": "BreadcrumbList",
    "itemListElement": [
      {
        "@type": "ListItem",
        "position": 1,
        "name": "Home",
        "item": "https://cloudguys.io/"
      },
      {
        "@type": "ListItem",
        "position": 2,
        "name": "Blog",
        "item": "https://cloudguys.io/blog"
      },
      {
        "@type": "ListItem",
        "position": 3,
        "name": "Idempotency Keys for Payment APIs: What \"Replay Byte for Byte\" Means",
        "item": "https://cloudguys.io/blog/idempotency-keys-money-apis"
      }
    ]
  },
  {
    "@context": "https://schema.org",
    "@type": "FAQPage",
    "@id": "https://cloudguys.io/blog/idempotency-keys-money-apis#faq",
    "mainEntity": [
      {
        "@type": "Question",
        "name": "Should the idempotency key be a random UUID?",
        "acceptedAnswer": {
          "@type": "Answer",
          "text": "Only if you generate it once per business operation and reuse it on every retry. A UUID generated per attempt means every retry moves money. Derive the key from the business object instead, such as transfer-{order_id}."
        }
      },
      {
        "@type": "Question",
        "name": "What happens if I reuse a key with a different body?",
        "acceptedAnswer": {
          "@type": "Answer",
          "text": "WalletD refuses it with 409 idempotency_key_reuse, because the stored SHA-256 request hash no longer matches. The IETF draft recommends 422 for the same case."
        }
      },
      {
        "@type": "Question",
        "name": "Does a 5xx mean no money moved?",
        "acceptedAnswer": {
          "@type": "Answer",
          "text": "No. The posting may have committed before the failure. Retry with the original request and key within a bounded budget, and never create a replacement payment under a new key."
        }
      },
      {
        "@type": "Question",
        "name": "How long does WalletD keep keys?",
        "acceptedAnswer": {
          "@type": "Answer",
          "text": "The public API's records have no automatic expiry today. ledgerd keeps completed keys for 30 days by default, after which a reused key would execute again. This is engineering guidance, not legal or compliance advice. Rules on holding customer funds, authorization windows and record keeping differ by country and licence, so check them with your counsel and regulator."
        }
      }
    ]
  }
]
```
