How to test payment idempotency

Where duplicate payments come from

A payment API receives a POST /payments, authorises the payment with the processor, and loses the connection before the client reads the response. The client retries. Without deduplication, the second request authorises and captures again, the payout settles the sale twice, and the customer sees two debits. RFC 9110 section 9.2.2 defines a request method as idempotent when multiple identical requests have the same intended effect as one, and lists PUT, DELETE, and safe methods as idempotent. POST is not on that list, and the same section says a client SHOULD NOT automatically retry a non-idempotent request unless it has some means to know that the request semantics are idempotent or that the original request was never applied.

The idempotency key is that means. The IETF Idempotency-Key header draft, version 07 of 15 October 2025, defines the header as a Structured Field String. The draft expired on 18 April 2026, is marked "Expired" in the httpapi working group, and is not an RFC (status checked 2026-09-22). Payment providers publish their own rules, and the rules differ, so every test starts from the documented behaviour of the API under test.

The consequence depends on the rail. On US consumer accounts, a duplicated electronic fund transfer fits the Regulation E error definition in 12 CFR 1005.11(a)(1)(ii): "an incorrect electronic fund transfer to or from the consumer's account". A consumer's notice of that error starts the investigation deadlines in section 1005.11(c), which our Regulation E page covers together with the evidence examiners ask for. On card rails, the second capture becomes a duplicate processing dispute; the acquiring segment page traces that money flow.

Procedure

1. Write down the documented rule per operation

List every state-changing operation: create payment, capture, refund, payout, transfer. For each, record the header name, key format, retention period, the response to a replay, and the response to a key reused with a different body. Providers answer these differently:

• The draft (section 2.6) says the resource SHOULD return the result of the completed operation, success or error, on a retry.
• PayPal's idempotency reference says a retry with the same PayPal-Request-Id returns "the latest status of the previous request", and that simultaneous requests with the same ID might fail the second one.
• GoCardless answers a reused key with an invalid_state error with reason idempotent_creation_conflict, and its Ruby client retrieves the existing record by default.

A client retry wrapper written for one of these behaviours mishandles the other two. The test oracle is the documented rule, and a missing rule is itself a finding: draft section 2.5.2 says resources MUST publish an idempotency specification, including the expiration policy where one applies.

2. Baseline request and exact replay

POST /v1/payments HTTP/1.1
Host: api.example.com
Authorization: Bearer sk_test_EXAMPLEONLY
Content-Type: application/json
Idempotency-Key: "5d2f6c0e-8b1a-4f37-9a0e-2c4b7d9e1f03"

{"amount_minor": 12500, "currency": "EUR", "payment_method": "pm_test_0001", "reference": "INV-2026-0042"}

Send it, record the response, then send the identical request again. Under the draft rule, both responses carry the same status and the same payment identifier:

{
  "id": "pay_000000000000000001",
  "status": "authorised",
  "amount_minor": 12500,
  "currency": "EUR",
  "reference": "INV-2026-0042"
}

A second id means the key is ignored. The same id with a different status can be correct under a "latest status" rule, so check against step 1.

3. Same key, altered body

Change amount_minor to 12600 and keep the key. Draft section 2.2 says the key MUST NOT be reused with a different payload, and section 2.7 says the resource SHOULD reply 422 with a link to documentation:

HTTP/1.1 422 Unprocessable Content
Content-Type: application/problem+json

{"type": "https://api.example.com/docs/idempotency", "title": "Idempotency-Key is already used"}

The defect to look for is a replay of the original response with status 201 and amount_minor 12500. A client that records its own request amount of 12600 without checking the replayed response books 126.00 EUR, the ledger shows 125.00 EUR, and the 1.00 EUR difference appears as a break in reconciliation.

4. Concurrent requests with one key

Sequential replays pass on implementations that check the key and write it in two separate steps. Parallel requests expose the gap:

import concurrent.futures, uuid, requests

URL = "https://api.example.com/v1/payments"
KEY = f'"{uuid.uuid4()}"'
BODY = {"amount_minor": 12500, "currency": "EUR",
        "payment_method": "pm_test_0001", "reference": "INV-2026-0043"}
HEADERS = {"Authorization": "Bearer sk_test_EXAMPLEONLY", "Idempotency-Key": KEY}

def send(_):
    r = requests.post(URL, json=BODY, headers=HEADERS, timeout=30)
    if "json" not in r.headers.get("Content-Type", ""):
        return r.status_code, None
    return r.status_code, r.json().get("id")

with concurrent.futures.ThreadPoolExecutor(max_workers=20) as pool:
    results = list(pool.map(send, range(20)))

ids = {pid for status, pid in results if status in (200, 201)}
assert len(ids) <= 1, f"duplicate payments created: {ids}"
assert all(status in (200, 201, 409) for status, _ in results), results

Draft section 2.7 says a retry while the original is still processing SHOULD receive 409. Run the script at least ten times per operation, because a race between key lookup and key insert does not fail on every run.

5. Timeout after processor approval

Configure the processor stub to approve the authorisation and then delay its response past the API's downstream timeout. Retry with the same key. The API must end with one authorisation at the processor and one payment record, whichever response the client receives. The payment flow testing service runs this fault injection per payment state.

6. Count side effects in the ledger

Responses can look correct while the ledger shows two postings. Query both the key and the business reference, because a client that generates a new key per retry defeats the key check completely:

-- duplicate postings under one idempotency key
SELECT idempotency_key, COUNT(*) AS postings, SUM(amount_minor) AS total_minor
FROM ledger_entries
WHERE entry_type = 'payment_debit' AND created_at >= :run_started_at
GROUP BY idempotency_key
HAVING COUNT(*) > 1;

-- duplicate payments for one business reference under different keys
SELECT reference, COUNT(DISTINCT payment_id) AS payments
FROM ledger_entries
WHERE entry_type = 'payment_debit' AND created_at >= :run_started_at
GROUP BY reference
HAVING COUNT(DISTINCT payment_id) > 1;

Both queries return zero rows on a passing run. The ledger reconciliation guide extends the second query to settlement files.

Pitfalls that vendor docs leave out

Three timing values decide whether a correct design double-charges. None of them appears in the draft; they follow from how key stores and processors interact.

In-flight lease shorter than the downstream timeout. Many implementations store the key with an "in progress" lease so a crashed worker does not block the key forever. For example, if the lease is 30 seconds and the processor call can take 60 seconds, a retry at second 45 finds an expired lease, takes the key, and sends a second authorisation. The lease must exceed the longest downstream timeout, multiplied by the internal retry attempts, plus the commit time. Test it by holding the stub response for lease plus 5 seconds and retrying inside that window.

Key retention shorter than the client's retry horizon. Draft section 2.3 leaves retention to the resource and only asks that it be published. For example, a job queue configured to redeliver a payout command for up to 72 hours outlives a key store that keeps keys for 24 hours, and the late redelivery creates a new payout. Retry once at retention minus one minute and once at retention plus one minute. After expiry, the API must reject the request or catch it by business reference, never create a second payment.

Stored transient errors. Draft section 2.6 returns the stored result "success or an error" on retry. If the API stores a processor timeout as a final 503, the payment can never complete under that key. If it stores nothing and the processor had approved, the retry charges twice. The correct state is "unknown", resolved by a status query to the processor before any new authorisation. Step 5 is the test that separates the three behaviours.

The Structured Field syntax adds one more trap. A client that follows the draft sends a quoted String, and a client written against an older provider example may send the bare value. A server that stores the raw header value treats "abc" and abc as two keys, so send both forms in one test.

Test matrix

The same cases apply to refunds, captures, and payouts, and to the webhooks each operation emits; the API and webhook testing service delivers them as an idempotency and retry behaviour report, and the webhook reliability guide covers duplicate event delivery.