Throughput and Latency Calculator
Apply Little’s Law to relate throughput, latency and concurrency.
Please note: An estimate based on the figures you entered. Real-world results vary with conditions this calculation cannot see, so treat it as a starting point rather than a final answer.
What the Throughput and Latency Calculator does
Little’s Law states that concurrency equals throughput times latency. It holds for any stable queueing system, which makes it the quickest sanity check in capacity planning: fix any two of the three and the third is determined.
Formula
L = λ × W (concurrency = throughput × latency)λ = L ÷ WW = L ÷ λServers = ⌈Throughput ÷ (Per-server capacity × Utilisation)⌉
Inputs explained
| Input | Unit | Required | Notes |
|---|---|---|---|
| Solve for | one of 3 options | Yes | — |
| Throughput | requests/sec | In some modes | Accepts more than 0. Shown Solve for is Concurrency from throughput and latency, or Solve for is Latency from concurrency and throughput. |
| Average latency | ms | In some modes | Accepts more than 0. Shown Solve for is Concurrency from throughput and latency, or Solve for is Throughput from concurrency and latency. |
| Concurrency | number | In some modes | Accepts more than 0. Shown Solve for is Throughput from concurrency and latency, or Solve for is Latency from concurrency and throughput. |
| Throughput per server | requests/sec | Optional | Optional — works out how many servers you need. Accepts 0 or more. |
| Target utilisation | % | Optional | Queues grow sharply above about 80%. |
How to use it
- Choose Solve for.
- Fill in the remaining inputs the form shows for your choice.
- Optionally add Throughput per server and Target utilisation.
- Select Calculate.
Worked example
A service handling 500 requests a second with 200 ms average latency.
- Throughput
- 500 req/sec
- Latency
- 200 ms
L = 500 × 0.2 = 100 requests in flight at any moment. That is the concurrency your connection pools and thread pools must support.
Frequently asked questions
Why does adding more threads stop improving throughput?
Because Little’s Law is a constraint, not a lever. Once the backend is saturated, extra concurrency simply raises latency proportionally and throughput stays flat.
What utilisation should I design for?
Around 70%. Queueing delay rises non-linearly with utilisation, so a system at 90% has dramatically worse tail latency than one at 70% for only modest hardware savings.