Surviving a traffic spike: how a waiting room keeps your site alive
Sometimes demand simply exceeds what your site can handle — a product drop, a viral moment, a ticket release. A waiting room is the difference between a queue and a crash.
A waiting room is a virtual queue that sits in front of your site. When more visitors arrive than you can safely serve, it admits people up to a capacity limit and holds the rest, showing their position and an estimated wait. Instead of everyone getting a broken site, the people inside get a working one and the queue moves steadily. The hard part is not the queue — it is choosing the capacity number.
There is a failure mode that caching does not solve. Caching works because most requests are for the same content; a waiting room exists for the situation where they are not — where thousands of people simultaneously need the genuinely dynamic parts of your site. Adding to a cart. Checking inventory. Paying.
You cannot cache your way out of that, because each of those requests is legitimately different and legitimately expensive. At that point you have exactly two options, and only one of them ends with sales.
With a waiting room
- Visitors admitted get a fully working site
- Everyone else sees a clear position and estimate
- Your origin stays inside its limits
- Sales continue throughout
Without one
- Everyone arrives at once
- The site slows, then times out
- Nobody completes a purchase, including the early arrivals
- Retries make it worse
The failure mode without a queue is worse than it first appears. Under overload, requests do not fail cleanly — they queue inside your application, time out after thirty seconds, and the visitor immediately refreshes, adding another request to a system already behind. A crash under load is self-reinforcing in a way a queue is not.
How the queue actually works
A visitor arrives and is checked
The edge looks for a valid session token. If they already have one, they pass straight through with no delay and no round trip to anything.
If there is room, they are admitted
Under the capacity limit, admission is immediate and invisible. Most visitors on most days never know the waiting room exists.
If not, they join the queue
They receive a queue page showing their position and an estimated wait, which polls for updates.
Slots free up and the queue advances
As active sessions end — through checkout, inactivity or leaving — the next people in line are admitted automatically.
How the queue knows someone has left
This is the part that determines whether a queue moves or stalls. If admission were only released when a session expired on a timer, a queue would advance at the speed of the timeout regardless of what people actually did.
In practice three signals free a slot. An admitted visitor's browser sends a periodic heartbeat, so the system knows they are still there. When they close the tab, the browser sends a departure beacon, releasing the slot immediately. And if heartbeats stop without a departure signal — a laptop lid closing, a phone losing signal — the session is released after an inactivity timeout. The first two are what make a queue feel responsive; the third is the safety net.
Choosing the capacity number
This is the genuinely hard part, and setting it wrong in either direction is costly.
Too high and the waiting room does nothing — you admit more people than the site can serve, everyone inside has a bad time, and you have added a queue to a broken site. Too low and you are queueing people unnecessarily, which is a real cost: some proportion of anyone asked to wait simply leaves.
The number should come from a load test, not from intuition. Find the concurrency at which your response times start climbing while throughput stops improving, then set capacity somewhat below that inflection point so there is headroom. Two additional considerations:
- Capacity means concurrent active sessions, not requests per second. A visitor idle on a product page consumes a slot while costing nothing.
- Session duration matters as much as capacity. With 500 slots and 10-minute sessions you admit 3,000 people an hour; halve the session length and you double the throughput — at the risk of ejecting someone mid-checkout.
- Bypass everything that is not a person shopping. Static assets, health checks, your own team's addresses, payment provider callbacks. A queue that catches your monitoring is a queue that blinds you exactly when you need to see.
Making the wait tolerable
Queue psychology is a real and well-studied thing, and it comes down to one finding: uncertain waits feel far longer than known ones. A queue that says "you are number 1,240 of 4,100, about six minutes" is experienced as dramatically shorter than the same wait with a spinner.
Which means the queue page deserves more thought than it usually gets. Show the position and update it visibly, so the number moving is proof the system works. Give an estimate and be conservative — an estimate that grows is far worse than one that shrinks. Explain briefly why the queue exists, since "high demand" is reassuring where an unexplained wait is alarming. And admit automatically rather than asking people to refresh, because someone who steps away should not lose their place.
Turn the waiting room on before the traffic arrives, not during. Enabling it while the site is already failing means the queue itself has to be configured under load, and the first thing a queue does is briefly increase requests as everyone's page reloads.
Frequently asked questions
What is a waiting room on a website?
A waiting room, or virtual queue, is a system that sits in front of a website and limits how many visitors are active at once. Visitors arriving while the site is below its capacity limit pass through immediately; those arriving when it is full are held on a queue page showing their position and estimated wait, and are admitted automatically as slots free up.
When should I use a waiting room?
When demand genuinely exceeds what your infrastructure can serve and the traffic cannot be cached away — product drops, ticket on-sales, limited-stock launches and viral moments. It is not a substitute for caching, which handles the far more common case of many people requesting the same content. It is for when thousands of people simultaneously need dynamic, per-person operations such as checkout.
How do I decide the capacity for a waiting room?
Derive it from a load test rather than intuition. Find the concurrency level at which response times begin climbing while throughput stops improving, and set capacity below that point to leave headroom. Note that capacity means concurrent active sessions rather than requests per second, and that session duration matters as much as the limit — shorter sessions move more people through, at the risk of ejecting someone mid-checkout.
How does a waiting room know when someone leaves?
Through three signals. Admitted visitors' browsers send periodic heartbeats confirming they are still present; closing a tab sends a departure beacon that releases the slot immediately; and if heartbeats stop without a departure signal, the session is released after an inactivity timeout. The first two keep the queue moving responsively, while the timeout catches closed laptops and lost connections.
Does a waiting room lose you customers?
Some proportion of queued visitors will leave, which is why capacity should not be set unnecessarily low. But the alternative under genuine overload is not a fast site — it is a site where requests time out, visitors refresh and make it worse, and nobody completes a purchase including those who arrived first. A queue converts some visitors; an overloaded site converts almost none.
Reserve it for the moments that warrant it — drops, on-sales, the day you unexpectedly end up on the front page of something — and a waiting room turns "the site went down" into "I waited two minutes and got mine". Which is a conversation your support team would very much prefer to have.
Mads has worked in IT — mostly hosting — since he was 16. He took an early stake in a SaaS company and helped grow it through to its acquisition by Visma, has built and run data-center networks, and served as CTO of a Danish data center. He started NordicCDN to make fast, secure infrastructure simple to use.
