Debugging & Real-World · Lesson 05

Idempotency in practice

A user double-clicks the "Pay" button. Your server's response is lost in transit and the client retries. Both scenarios send the same request twice — and without idempotency, both cause a double charge. Idempotency keys turn "retry safely" from a hopeful claim into a mechanical guarantee.

The problem: two sends, two effects

An operation is idempotent if applying it multiple times produces the same result as applying it once. GET, PUT, and DELETE are designed to be idempotent. POST is not — two identical POST requests to /charges are intended to create two charges. That's fine in a world with perfectly reliable networks, but networks are not perfectly reliable.

Consider this sequence:

The same failure mode appears in other shapes: a "Pay" button with no loading state that a user clicks twice before seeing a response; a background job that retries on any error, not just safe ones; a webhook that fires twice for the same event.

The fix: idempotency keys end-to-end

An idempotency key is a client-generated unique identifier attached to a request. The server uses it as a deduplication key: the first request with a given key is processed and the result is cached; any subsequent request with the same key returns the cached result without re-executing the operation.

The key property: the client generates the key before sending the request, and reuses the same key on retries. This way, even if the response was lost and the request is sent again, the server recognizes it as a repeat and returns the cached response — the charge is not re-executed.

How idempotency keys work over HTTP

The client sends the key in a request header. Stripe uses Idempotency-Key; PayPal uses PayPal-Request-Id; other providers may use X-Idempotency-Key. The value should be a UUID (version 4) generated fresh for each logical operation — not for each retry of the same operation.

Implementing the dedup store

The server-side implementation requires a store that maps idempotency keys to cached responses. The store must persist at least as long as the retry window you want to support (24–48 hours is common). Redis with a TTL works well; a database table also works. The critical operations:

1. On receiving a request: look up the idempotency key in the store. If found, return the cached response immediately — do not re-execute the operation.
2. If not found: execute the operation (charge the card, create the record), capture the response, write the key + response to the store, then return the response.
3. Handle the in-flight race: if two requests with the same key arrive simultaneously, one should execute and the other should wait for the result. Use a database lock or a Redis SET NX (set if not exists) to prevent concurrent processing of the same key.

Idempotent webhook consumers

Webhook handlers are the consumer side of the same problem. As covered in Lesson 04, the provider delivers at-least-once — your handler may process the same event twice. The idempotency key in this case is the event's unique ID provided by the provider.

Idempotency and the request body must match

What happens if a client sends the same idempotency key with a different request body? For example, key abc123 first with amount: 4900, then with amount: 2500? The server should return an error — the key is associated with a specific operation, and allowing the body to change would make the guarantee meaningless. The standard response is 422 with an error code like idempotency_key_in_use.

Clients can use this as a debugging signal: if they're getting idempotency_key_in_use errors, they're accidentally reusing keys across different operations — a bug in how they generate keys.

Tying it together: end-to-end idempotency

A fully idempotent payment flow looks like this:

1. Client generates a UUID before the user clicks "Pay." Store it associated with the user's current checkout session.
2. Client sends the charge request with the UUID in the Idempotency-Key header.
3. Server checks the dedup store before executing the charge. If the key exists, return the cached response.
4. If new: charge the card, cache the result with the key (both success and card errors), return the response.
5. On network failure: the client retries with the same key. The server returns the cached result. The card is not charged again.
6. Payment provider sends a webhook confirming the charge. Your webhook handler deduplicates by event ID in the same database transaction as the order status update.
7. Result: regardless of how many times the request was sent (double-click, network retry, webhook retry), the user is charged exactly once.

Under the hood: how it actually works

Idempotency keys look simple from the outside — "send a header, get dedup." The mechanism that makes it safe under concurrent duplicates is the interesting part.

The dedupe store: keyed by idempotency key, atomic insert-if-absent

The server maintains a store (Redis or a database table) whose schema is conceptually:

-- Database table equivalent
CREATE TABLE idempotency_keys (
    key        TEXT        PRIMARY KEY,       -- the UUID the client sent
    status     TEXT        NOT NULL,          -- 'pending' | 'complete'
    request_hash TEXT      NOT NULL,          -- hash of the request body (body-mismatch guard)
    response   JSONB,                         -- cached {status_code, body}
    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    expires_at TIMESTAMPTZ NOT NULL           -- TTL: typically now() + 48 h
);

Atomic insert-if-absent: how SET NX prevents races

The critical invariant is: exactly one thread must execute the underlying operation, even when two requests with the same key arrive simultaneously. A plain "check then execute" is a race — both threads can read "key not found" and both proceed to charge the card. The fix is an atomic insert-if-absent:

• Redis: SET idem:{key} "pending" NX EX 172800 — "set only if Not eXists, with a 48-hour expiry." This is a single atomic command. Only one concurrent caller wins; the other gets nil back.
• PostgreSQL: INSERT INTO idempotency_keys … ON CONFLICT (key) DO NOTHING followed by checking the row count — rows-affected = 0 means another request already claimed the key.

The losing concurrent request sees the key in pending state. The correct behaviour is a short-circuit: return a 409 Conflict (or optionally poll and wait a few seconds for the pending → complete transition). This is safer than blocking indefinitely — the winning request may have crashed, leaving the key stuck at pending without a result.

Storing and replaying the first response

After the operation completes (success or deterministic error), the server overwrites the pending row with the full response — both the status code and the body. Subsequent requests look up the key, find it complete, and replay the stored response verbatim. The client cannot tell whether it received a live response or a cached replay — that is the guarantee. The stored body must include everything the client needs (charge ID, amount, status), not just a "seen" flag.

The TTL trade-off

The TTL controls how long the server remembers a key. Longer TTLs give clients more time to retry after extended outages but consume more storage. The practical formula: TTL should exceed the longest plausible retry window for your clients. For a payment API, 24–48 hours covers mobile apps that go offline and retry next morning. After the TTL, the key is evicted — a retry with the same key would be treated as a fresh request. This is why clients must not reuse keys across sessions: the key should be tied to one logical operation and discarded after success.

Traced duplicate: two simultaneous POSTs, one charge

• Stripe — Idempotent requests (including pending-key behaviour)
• Redis SET NX — atomic insert-if-absent documentation
• AWS Builder's Library — Making retries safe with idempotent APIs

Sources & further reading

• Stripe — Idempotent requests
• MDN — Idempotent (HTTP methods)
• AWS Builder's Library — Making retries safe with idempotent APIs
• RFC 9110 §9.2.2 — Idempotent Methods