ENGINEERING NOTES / FINTECH ENGINEERING
Idempotency Keys for Payment APIs: What "Replay Byte for Byte" Means
An idempotency key makes a timed-out payment safe to retry. What exactly-once really requires in the database, why the replay should be byte-identical, what happens across two services, and how Stripe, Adyen and the IETF draft differ.

An idempotency key lets a client retry a money-moving request without moving the money twice: the first request with a given key runs, and every later request with the same key gets the first response back instead of running again. “Replay byte for byte” means that returned response is the identical status code and identical body bytes, not a freshly rendered copy, so a client that compares, hashes or logs responses sees exactly one outcome.
The header is the easy part. What takes work is making “exactly once” true when two duplicates arrive at the same instant, when the server crashes between steps, and when the money lives in a different service from the request. Below is how the WalletD API handles each case, with the code and tests behind every claim, and how that compares with the published behaviour of Stripe, Adyen and the IETF draft. CloudGuys builds WalletD. Facts were checked on 8 November 2026.
The short answer
- Require the key on every money-moving
POST. Reads and configuration changes do not need one. - Store the key in the same database transaction as the money. A key that commits separately from the work it guards is a race.
- Hash the request and refuse a reused key with a different body (WalletD answers
409 idempotency_key_reuse). - Return the stored bytes on replay, including to a duplicate that arrived while the first was still running.
- Derive keys from your business object, such as
payment-order-4471, never a fresh UUID per attempt. - Across services, derive downstream keys from the upstream one, and let the caller ask the downstream service what already happened.
What does an idempotency key actually guarantee?
One logical operation has at most one effect, however many times the request is sent. A network can fail after the server acted: the transfer happened, the response was lost, and the client cannot tell. With a key, the client resends with the same key and gets either the stored result or, if nothing committed, a first execution.
That only works if the client keeps the key stable. WalletD’s developer docs name the failure modes: a uuid4() generated per attempt, or a timestamp key, means “every retry moves money”, and a constant key like transfer means your second, unrelated transfer is refused as a replay. Good keys come from the business object: payout-invoice-8891, transfer-{order_id}. Keys are scoped per tenant, and merchant-bound API keys add no merchant namespace, so a marketplace should put the merchant in the key (merchant-42-transfer-order-8891).
How do you make it exactly-once in the database?
Insert the key row first, inside the same transaction that does the work, and let the unique index arbitrate. WalletD’s arbiter (internal/idempotency/idempotency.go, recorded as ADR-0006) does four things:
INSERT INTO idempotency_keys (tenant_id, key, request_hash) ... ON CONFLICT DO NOTHING, then run the operation in the same transaction, then store the response code and body on the key row.- A concurrent duplicate’s insert blocks on the winner’s uncommitted row in PostgreSQL. If the winner commits, the duplicate’s insert affects zero rows and it reads the stored response. If the winner rolls back, the duplicate’s insert proceeds and it becomes the winner. The arbiter tries this up to three times.
- If the stored
request_hash(SHA-256 of the request) differs from the new one, it refuses withmoney.ErrIdempotencyReuse, which the API maps to409 idempotency_key_reuse. - A failed operation rolls back the key row too, so a retry after a refusal can succeed.
The core of it, trimmed from the source:
tag, err := tx.Exec(ctx,
`INSERT INTO idempotency_keys (tenant_id, key, request_hash)
VALUES ($1, $2, $3) ON CONFLICT DO NOTHING`,
tenantID, key, requestHash)
if tag.RowsAffected() == 0 {
return errLostRace // someone else holds the key: read their stored result
}
result, err = fn(tx) // the money movement, same transaction
// RETURNING hands back the jsonb-normalized bytes so the winner's
// response is byte-identical to every future replay.
return tx.QueryRow(ctx,
`UPDATE idempotency_keys SET response_code = $1, response_body = $2, txn_id = $3
WHERE tenant_id = $4 AND key = $5 RETURNING response_body`,
result.StatusCode, body, result.TxnID, tenantID, key,
).Scan(&result.Body)
Point 2 is the part to copy first: duplicates wait for the winner instead of getting an error, so the client never has to resolve a race.
Why should the replay be byte-identical?
Because “the same result” is ambiguous, and clients depend on the bytes. If the server re-renders a replay from current state, the body can change: a status moved on, a field was added, numbers were reformatted. A client that logs the first response, or a webhook consumer that deduplicates on a body hash, then sees two different outcomes for one operation.
WalletD stores the response in a PostgreSQL jsonb column, and the winner returns the bytes PostgreSQL hands back through RETURNING rather than the bytes it rendered. jsonb normalises whitespace and key order, so the very first response and every replay are the same bytes. A test asserts it: TestIdempotentReplayIsByteIdentical.
What happens when the money lives in another service?
You can no longer put the key and the money in one transaction, so you derive every downstream key from the upstream one and let the caller ask what already happened.
WalletD’s books live in a separate ledger service, ledgerd, with its own database. No transaction spans the two. The failure to design for is: ledgerd commits the posting, then walletd crashes before it records the payment row and the stored response. The retry must find that posting, not make a second one.
Three mechanisms close that window:
- Deterministic ids. The payment id is a UUIDv5 of the tenant and the client’s key, and the ledger posting key derives from it (
pay:<payment id>:capture,refund:<refund id>). A retry rebuilds the same posting request, and ledgerd’s own idempotency layer replays it. - A stored-result probe. ledgerd exposes
GET /v1/ledgers/{id}/idempotency/{key}, which returns the stored result with no request-hash check, because the repairing caller may not be able to rebuild the original request (the fee schedule may have changed in between). A miss is authoritative inside the retention window: the posting never committed. - Repair from the stored legs. When a retry’s rebuilt request no longer matches, walletd recovers the committed posting and finishes its own records from those amounts, because the stored legs are the money that moved (ADR-0022). That ADR records eight money paths where the first version of this recovery was broken, all the same way: the retry re-checked mutable state the money had already left and refused before reaching the replay.
The practical consequence for an integrator, stated in the docs: a 5xx does not prove that no money moved. Keep the original request and key, retry within a bounded budget, and never create a replacement payment with a new key.
How long should keys live?
Long enough to cover every realistic retry, with the policy published. The IETF draft says a resource “SHOULD define such expiration policy and publish it in the documentation” (draft-ietf-httpapi-idempotency-key-header-08, 22 October 2026).
In WalletD the two layers differ, and the docs say so. The public API’s idempotency records currently have no automatic expiry or cleanup. ledgerd keeps completed keys for a configurable retention, 30 days by default, purged by a daily job; after the purge a reused key would execute again. Retries arrive in seconds or minutes, so the repair window sits well inside it.
How do Stripe, Adyen and the IETF draft compare?
Their published behaviour, read on 8 November 2026. We did not test the third-party APIs for this post; every cell outside the WalletD column comes from the linked documentation.
| WalletD API | Stripe (API v1) | Adyen | IETF draft-08 | |
|---|---|---|---|---|
| Header | Idempotency-Key, required on money-moving POST |
Idempotency-Key |
idempotency-key |
Idempotency-Key |
| Key limits | Scoped per tenant | Up to 255 characters | Up to 64 characters, unique per company account | Defined by the resource |
| Same key, different body | 409 idempotency_key_reuse |
Error: parameters are compared with the original | Not stated on the page we read | 422 recommended |
| Duplicate while first is running | Waits, then gets the stored response | Not saved; you can retry | 422 or 409, error code 704 |
409 recommended |
| What a replay returns | Stored status and body, byte-identical | Saved status and body, including 500s |
The first response | Defined by the resource |
| Retention | No automatic expiry at the API; ledger keys 30 days | Keys can be pruned after at least 24 hours | 7 to 14 days | Resource must publish its policy |
Sources: Stripe idempotent requests, Adyen API idempotency, IETF draft-08. The draft is still an Internet-Draft, a work in progress, not an RFC.
Two differences matter in practice. WalletD returns 409 for a mismatched body where the draft recommends 422, so if you build to the draft, handle both. And Stripe and Adyen document a concurrent duplicate as something the client deals with (retry, or a 409/422 error), while WalletD makes the duplicate wait for the winner. Neither approach is wrong, but your retry loop has to know which one it is talking to.
What did we verify ourselves?
We ran the relevant WalletD suites on 8 November 2026 against disposable PostgreSQL containers and a real ledgerd started by the test harness (walletd at f446353 with uncommitted working-tree changes in unrelated catalog files; ledgerd at acf0cba). All passed, among them:
--- PASS: TestIdempotentReplayIsByteIdentical (8.06s)
--- PASS: TestIdempotentRejectsDifferentPayload (0.05s)
--- PASS: TestIdempotentReleasesKeyOnFailure (0.04s)
--- PASS: TestIdempotentConcurrentDuplicatesRunOnce (0.05s)
--- PASS: TestLedgerdInstantPaymentWithFeeAndCommission (7.97s)
--- PASS: TestLedgerdVoidRefusesAndCaptureRepairsAfterLostCaptureCommit (0.06s)
TestIdempotentConcurrentDuplicatesRunOnce sends 20 concurrent top-ups of 7,777 under one key and requires one transaction, a balance of exactly 7,777, and identical bodies for all 20 callers. The lost-commit test posts the ledger half by hand under the keys walletd derives, then runs the retry and asserts a single debit. In the ledger pillar post we describe the chaos gate, which adds 50 keys hit by four concurrent callers each inside 1,000 mixed operations.
A checklist for your own money API
- Make the key mandatory on every money-moving endpoint and reject requests without it before doing anything.
- Insert the key row inside the business transaction; never check-then-insert in two steps.
- Store a request hash and refuse reuse with a different body, with a stable error code.
- Store and return the response bytes; do not re-render replays.
- Release the key when the operation fails, so a corrected retry can run.
- Derive every downstream key and id deterministically from the client’s key, and give callers a way to ask what committed.
- Publish the retention period and the status codes. Clients write retry loops against your documentation, not your code.
FAQ
Should the idempotency key be a random UUID?
Only if you generate it once per business operation and reuse it on every retry. A UUID generated per attempt means every retry moves money. Derive the key from the business object instead, such as transfer-{order_id}.
What happens if I reuse a key with a different body?
WalletD refuses it with 409 idempotency_key_reuse, because the stored SHA-256 request hash no longer matches. The IETF draft recommends 422 for the same case.
Does a 5xx mean no money moved? No. The posting may have committed before the failure. Retry with the original request and key within a bounded budget, and never create a replacement payment under a new key.
How long does WalletD keep keys? The public API’s records have no automatic expiry today. ledgerd keeps completed keys for 30 days by default, after which a reused key would execute again.
This is engineering guidance, not legal or compliance advice. Rules on holding customer funds, authorization windows and record keeping differ by country and licence, so check them with your counsel and regulator.
Try it on a sandbox
WalletD requires an idempotency key on every money-moving call and documents the retry rules in six languages. It is in early access, with no public pricing; the WalletD product page has the overview. To see the replay behaviour against a sandbox tenant, request a WalletD walkthrough.
BUILT AND OPERATED BY CLOUDGUYS
WalletD: wallet infrastructure
Wallet infrastructure for business, on a ledger you can audit.
Merchant collections, balances, payments, rewards and payouts behind one API, with every movement posted to a separate double-entry ledger that reconciles by construction. Run it on your own infrastructure or in the cloud.
Visit WalletD ↗ About WalletD ↗