Throughput and Latency Calculator
Apply Little’s Law to relate throughput, latency and concurrency.
Please note: An estimate based on the figures you entered. Real-world results vary with conditions this calculation cannot see, so treat it as a starting point rather than a final answer.
What the Throughput and Latency Calculator does
Little’s Law states that concurrency equals throughput times latency. It holds for any stable queueing system, which makes it the quickest sanity check in capacity planning: fix any two of the three and the third is determined.
Formula
L = λ × W (concurrency = throughput × latency)λ = L ÷ WW = L ÷ λServers = ⌈Throughput ÷ (Per-server capacity × Utilisation)⌉
Inputs explained
| Input | Unit | Required | Notes |
|---|---|---|---|
| Solve for | one of 3 options | Yes | — |
| Throughput | requests/sec | In some modes | Accepts more than 0. Shown Solve for is Concurrency from throughput and latency, or Solve for is Latency from concurrency and throughput. |
| Average latency | ms | In some modes | Accepts more than 0. Shown Solve for is Concurrency from throughput and latency, or Solve for is Throughput from concurrency and latency. |
| Concurrency | number | In some modes | Accepts more than 0. Shown Solve for is Throughput from concurrency and latency, or Solve for is Latency from concurrency and throughput. |
| Throughput per server | requests/sec | Optional | Optional — works out how many servers you need. Accepts 0 or more. |
| Target utilisation | % | Optional | Queues grow sharply above about 80%. |
How to use it
- Choose Solve for.
- Fill in the remaining inputs the form shows for your choice.
- Optionally add Throughput per server and Target utilisation.
- Select Calculate.
Worked example
A service handling 500 requests a second with 200 ms average latency.
- Throughput
- 500 req/sec
- Latency
- 200 ms
L = 500 × 0.2 = 100 requests in flight at any moment. That is the concurrency your connection pools and thread pools must support.
Reading the result
- "Servers at X% utilisation" against plain "servers needed" shows the real trade-off between running lean and keeping burst headroom — "extra servers for headroom" quantifies exactly what that safety margin costs.
- The two sensitivity tables answer complementary questions: one shows how much concurrency capacity is needed as latency varies, useful for connection-pool sizing; the other shows how throughput scales with worker count at a fixed latency, useful for capacity planning.
Assumptions and limitations
- Little's Law describes average, steady-state behaviour — a system sized exactly to the computed average concurrency will still see moments above it, which is exactly why the utilisation-target feature exists.
- "Throughput per server" is treated as one flat, identical figure across every server — uneven load distribution in a real deployment (a naive load balancer, hot keys) means some servers run hotter than this uniform assumption implies.
Common mistakes
- Sizing infrastructure for very high utilisation to save cost — the notes warn queueing delay rises sharply above roughly 80%, so a system with a fine-looking average latency at high utilisation can still have much worse tail latency for the requests that matter most to users.
Frequently asked questions
Why does adding more threads stop improving throughput?
Because Little’s Law is a constraint, not a lever. Once the backend is saturated, extra concurrency simply raises latency proportionally and throughput stays flat.
What utilisation should I design for?
Around 70%. Queueing delay rises non-linearly with utilisation, so a system at 90% has dramatically worse tail latency than one at 70% for only modest hardware savings.