June 4, 2026, Luboš Zápotočný
Why your checkout collapses at 10% server load
Servers idle, checkout down. Lock contention, cache stampedes, and synchronous payment calls: the failure modes autoscaling does not address.
It’s the most confusing incident an e-commerce team can have. The campaign goes live, checkout starts timing out, and the dashboards insist nothing is wrong: CPU at 10%, memory fine, plenty of headroom. The instinct is to add servers. The servers arrive, idle alongside the others, and checkout is still down.
The confusion comes from a wrong mental model. Load has two components: work and waiting. Your servers are queueing rather than working, and queues don’t show up on a CPU graph.
The common causes
Database lock contention. Checkout is where the shop stops reading and starts writing: reserve stock, create the order, save the address. Those writes serialize on row locks, and campaign traffic concentrates on the same few rows. A hundred customers checking out during the same promotion serialize on the same hot rows: the order-sequence counters, the order and quote tables, the stock being reserved. Each transaction holds its locks while it waits on other work. Throughput collapses while CPU stays low. On Magento this has a well-known signature: the lock-wait and deadlock messages that fill the logs the morning after a sale.
Cache stampedes. A hot cache entry (the category page every ad points at) expires under peak traffic. Every request that misses now regenerates the page simultaneously, issuing identical expensive queries against the database. The whole sequence is triggered by nothing more than a TTL expiring at the wrong moment.
Synchronous third-party calls in the checkout path. Shipping rates, payment authorization, tax calculation, an anti-fraud check: each a network call to someone else’s system, made while your worker holds a connection and often a database transaction. When one provider slows from 200 ms to 10 seconds under the same campaign load, every worker gradually ends up blocked inside that call. The timing of your outage is now set by a vendor you don’t control.
Worker and connection exhaustion. The three problems above share a common amplifier: workers that wait still occupy their slot and their database connection. PHP-FPM processes sized for ordinary weekday traffic fill with blocked requests, new requests can’t get a worker or a connection at all, and now everything that still needs a PHP worker is down, however little work that page does.
None of the four is a hardware problem, which is why autoscaling doesn’t resolve it: new instances join the same queue for the same row lock behind the same slow vendor. The added capacity waits in parallel instead of adding throughput.
How to see it coming
The diagnostic shift is from utilization to time: for a checkout request, where do its milliseconds go? Working, or waiting on locks, caches, and third-party APIs? Distributed traces answer this directly; a slow-query log plus the database’s lock-wait counters answer it cheaply. If you only add one graph before the next campaign, make it lock waits.
The fixes follow from the diagnosis, and most are routine: timeouts on every third-party call plus cached fallbacks wherever a stale answer is safe (a shipping rate a few minutes old does no harm, though a payment authorization must still go live), stampede protection so one process rebuilds a hot key while others serve stale, moving non-essential writes out of the transaction and into a queue, and reserving stock as late in the funnel as the business allows rather than holding it from the cart. Which of these matters for your shop is an empirical question. A performance audit answers it from your own traces, and our DevOps work keeps the fixes in place.
Most of these incidents turn out to be concurrency problems, so start with a trace before you buy more servers.