Caching

A database query may parse or reuse a plan, read pages from memory or storage, take locks, and hold a connection for the trip. Its cost ranges from sub-millisecond key lookups to seconds of scanning, and concurrency is finite.

A cache lookup is usually a simpler in-memory operation over the network. Depending on payload, hardware, and pipelining, one cache node can serve a very high request rate with much lower per-read work than the origin.

Caching improves both latency and capacity by moving repeated reads off a scarce authoritative store and onto a cheaper serving layer. Which benefit matters more depends on the workload.