Queueing theory
A queue is work waiting for a resource. Customers at a checkout, questions for a colleague and jobs for a computer all form queues. When arrivals are uneven, a little more demand can cause a lot more waiting.
Change how busy it is.
One support person takes six minutes per request on average, so they can handle ten an hour. Drag the tall bar up or down to change their utilisation, the share of time they are busy.
Drag vertically. Up and down arrows adjust utilisation; Home sets zero and End sets 100 percent.
80 percent busy divided by 20 percent spare capacity, times 6 minutes of service, equals 24 minutes of average waiting.
Spare capacity lets a queue clear between bursts. As utilisation approaches 100%, the long-run average wait in this model grows without bound. At 100%, there is no finite steady-state average. A real system eventually runs out of space, turns people away, or loses them to the wait.
What this model assumes
The example uses an M/M/1 queue. Requests arrive independently at a constant average rate, following a Poisson process. One person handles them first in, first out (FIFO), with independent, exponentially distributed service times. The model assumes unlimited waiting space and no one leaving early. The curve shows a long-run average, not a promise for an individual request or a forecast for the next hour.
FIFO is one way to organise a queue. Emergency care often gives priority to urgent cases, while appointments reserve time in advance. People may still have to wait when the resource they need is busy.
Perfectly regular arrivals and identical service times can run at exactly 100% with no wait if they line up. Randomness is what makes spare capacity so important in this example. There is no universal “safe” utilisation percentage.
MIT’s queueing models notes explain the notation, queue disciplines, and stability conditions.
Change the pattern.
A distribution describes the possible values and how likely they are. Drag the blue handles to move each pattern or change its spread. You can also use the arrow keys. Each chart is a separate example.
Drag a handle horizontally. Arrow keys adjust the value; Shift makes a larger change. Home and End move to the limits.
Poisson
Exponential
Fixed
Normal
Lognormal
Erlang
Poisson bars give probabilities for whole-number counts. For a continuous curve, probability is the area over an interval; the height at one point is a density.
Where these patterns apply
Poisson counts and exponential gaps describe two views of the same constant-rate, independent arrival process. At eight arrivals an hour, the average count in fifteen minutes is two, and the average gap is 7.5 minutes. Outages, launches, and scheduled batches can break those assumptions.
Standard deviation measures how spread out the durations are around their mean. A larger value means more variation.
Fixed durations suit timed machine cycles. Normal distributions can approximate small, symmetric timing errors. They also allow negative values, which makes them a poor fit for some durations. The normal chart keeps its mean at six minutes while you change the standard deviation.
Lognormal durations are always positive, with a long right tail. They may fit work such as investigations or repairs, but you need to check against measurements. The chart keeps its mean at six minutes while you change the standard deviation. Some outcomes extend beyond the visible range.
Erlang describes the sum of independent exponential stages with the same rate. It is a gamma distribution with a whole-number shape and can model a job with several similar stages. Here the total mean stays at six minutes, so adding stages makes each one shorter and the total more consistent. One stage is exponential.
All the controls change theoretical distributions. Choose a model by checking measured arrivals and durations, including bursts and unusually long jobs.
Long jobs make the queue longer.
Keep eight arrivals an hour and a six-minute average service time. Change only how much service times vary. A long job holds up everyone behind it, so the same average capacity can produce very different waits.
Preparing requests before handing them over can reduce avoidable variation. So can using appointments where appropriate, or separating very long jobs from quick ones. Measure the effect on everyone who waits, since giving one group priority can delay another.
The relationship between variability and waiting
For an M/G/1 queue, arrivals are Poisson and service times can have any distribution with a finite second moment. The Pollaczek–Khintchine formula below gives the average wait when service times are independent, one server works in FIFO order, and utilisation is below one.
Wq = [(1 + Cs²) / 2] × [ρ / (1 − ρ)] × E[S]
Cs is the service-time standard deviation divided by its mean. E[S] is mean service time. Fixed, exponential, and this lognormal example have Cs values of 0, 1, and 2. Their waiting-time multipliers are ½, 1, and 2½. At its initial one-minute spread, the normal example has Cs ≈ ⅙ if its negligible negative tail is removed.
See Larson and Odoni’s treatment of the M/G/1 queue. This formula does not model correlated bursts, multiple servers, or customers leaving.
A chain has a narrowest point.
A bottleneck is the step whose capacity limits the whole process. Here, every request passes through intake, review and reply, with one person at each step. Review starts at eight requests an hour, so the chain can handle at most eight, even if the other steps are faster. Drag the bars up or down to change each step’s capacity.
Drag a bar vertically or use the up and down arrow keys. Home sets zero and End sets twenty requests an hour. Darker bars show the bottleneck.
At the starting settings, speeding up intake leaves the limit unchanged. Speeding up review raises it until another step becomes the bottleneck. The darker bars show which steps set the limit. Actual throughput may be lower, depending on demand, variation and whether work can reach each step.
Count the work that is waiting.
Little’s law says that the average number of items in a system equals its throughput multiplied by the average time spent there. These are long-run averages, and arrivals do not have to follow a Poisson process.
If a stable team completes eight requests an hour and each request spends half an hour waiting and being worked on, there are four requests in the system on average. At the starting settings of the first example, 3.2 are waiting and 0.8 are in service.
The queueing formulas
Let λ be arrivals per hour, μ be one person’s service capacity per hour, and ρ = λ / μ be utilisation. For the unbounded M/M/1 model, λ must be less than μ.
Average wait: Wq = λ / [μ(μ − λ)]
Average time including service: W = 1 / (μ − λ)
Average number waiting: Lq = λWq = ρ² / (1 − ρ)
Average number in the system: L = λW = ρ / (1 − ρ)
These times are in hours. Multiply by 60 for minutes. With λ = 8 and μ = 10, Wq = 0.4 hours and W = 0.5 hours.
Little’s law uses the same boundary for the count, throughput, and time. For a queue alone, Lq = λWq. If some arrivals are rejected, use the rate admitted to the system. Long-run averages must exist, with flow balanced over the period of interest.
Karl Sigman’s notes on Little’s law explain the conditions. The DIMACS formula sheet covers more models. When using it, check whether a formula counts only the queue or includes people being served, and whether its time includes service.
Decide what happens when it fills up.
A queue can absorb a burst, but it keeps growing if demand stays above capacity. You have three options.
- Increase capacity. Add help at the bottleneck, reduce the work needed per request, or remove a slow step. Leave room for the bursts you expect.
- Decrease demand. Prevent repeat problems, remove unnecessary requests, or offer a useful self-service route. Spreading arrivals out can also reduce peaks, even when total demand stays the same.
- Limit the queue. Decide how much waiting you will accept and what happens when the queue is full. You could decline new work, offer a later slot or expire requests that are no longer useful.
Limiting the queue does not increase capacity, and it may move the wait somewhere else. A “try again later” response can create more traffic if everyone retries together. Track rejections and abandoned requests alongside the waiting time of the people you serve.
I find queueing theory useful because it shows the cost of keeping everything fully booked. Leaving some time unallocated gives the next unexpected request somewhere to go.