Databases Under Load
Why the database goes first
Application replicas are often cheap and replaceable. A database owns durable state and is harder to replace or partition, which is why so many capacity problems eventually resolve to the same node on the canvas.
A database has two independent ceilings, and hitting either one looks like an outage. The first is throughput — how many queries per second it can actually execute, set by CPU, disk, and how expensive your queries are. The second is concurrency — how many connections it will hold open at once.
The second ceiling is the one that surprises people. If each query occupies a connection for 25ms, Little's Law gives a theoretical pool ceiling of 40,000 queries per second for 1,000 connections, or 4,000 for 100. Other bottlenecks usually lower that ceiling, and making the pool larger than the database can execute concurrently can make contention worse rather than add capacity.
pool_limited_qps ≤ pool_size / connection_hold_seconds 1000 conns / 0.025 s = 40,000 q/s 100 conns / 0.025 s = 4,000 q/s 1000 conns / 0.250 s = 4,000 q/s An upper bound, not a throughput promise.
Halving your query time doubles your effective connection pool. Query optimisation is capacity work.
3 components2 connections0:00
Recording…