Webhooks retry. Your integration should assume it.

Read the webhook documentation of any serious provider (Stripe, Shopify, Adyen, GoPay) and you’ll find the same warning in one form or another: deliveries may occur more than once. Within the retry window the contract is at-least-once, not exactly-once; if your endpoint stays down for the whole window, the event arrives zero times, which is why a reconciliation job that polls the provider’s API is the backstop underneath everything else. Exactly-once delivery over an unreliable network is not achievable, so providers take the safe option: when they cannot confirm you received an event, they send it again.

Most integrations are written as if that clause didn’t exist. They work in the demo, they work in month one, and then a timeout happens somewhere between the provider and a 200 response, and the same order.paid event arrives twice.

What double delivery actually breaks

• The customer receives two order confirmations.
• Stock gets decremented twice, and incorrect “out of stock” listings start appearing.
• An invoice is issued twice, and accounting finds it months later.
• A loyalty balance is credited twice; nobody notices until the numbers stop reconciling.

The provider did what it documented; the handler assumed a guarantee the provider never made.

Idempotency is the fix

An idempotent handler produces the same end state whether an event is processed once or five times. That single property absorbs retries, manual replays, and a class of race conditions. Achieving it is mostly discipline:

1. Key every event. Providers send an event ID; if yours doesn’t, derive one from stable fields. Record processed IDs and skip duplicates, and enforce that with a unique constraint in the database rather than in application code: the application check has a race window; the constraint doesn’t.
2. Prefer absolute state over relative changes. “Set status to paid” is naturally idempotent. “Decrement stock by 2” is not. When the provider sends deltas, translate them into upserts against your own record of the event, not unconditional arithmetic.
3. Assume disorder, too. Retries also arrive out of order: order.updated can land before order.created. Version numbers or timestamps on the target record (“apply only if newer”) handle what sequence assumptions can’t.

The rest of the checklist

Idempotency is the core; three habits around it keep an integration out of the ticket queue:

• Acknowledge fast, process async. Verify the signature, persist the event, return 200, and do the real work from a queue. Slow synchronous handlers cause the timeouts that cause the retries you were worried about.
• Authenticate every webhook. A webhook endpoint is an unauthenticated URL on the public internet that mutates your business state. Most providers sign the payload with an HMAC you recompute and compare; some (GoPay) give you only an ID and expect you to re-fetch the authoritative state from their API. Either way, never act on the raw request as received.
• Keep a dead-letter queue with a replay tool. Some events will fail for reasons retry can’t fix: a deleted SKU, a mapping gap. Route them there, alert on them, and make it possible to replay a single event without SSH-ing into production. When a provider has an outage lasting hours, replay lets you catch up afterwards without declaring an incident.

One test covers all of the above: take yesterday’s events from staging and deliver every one of them twice, in shuffled order. If the end state matches production, your handlers really are idempotent.

This is the standard we build integrations to. It is most of what our integrations service page describes, and the same thinking shows up in our custom backend work. And if your stock numbers already drift, the webhook layer is one of the first places to look.