Thread Pools — Reusing Threads for Task Execution

Why it matters

Creating threads per request destroys server performance. Thread pools are the default for any real concurrent server.

The architecture

Pool: fixed or bounded number of worker threads. Queue of pending tasks.

Workers take tasks, execute, return to pool. Reused across many tasks.

How it works end to end

Sizing: CPU-bound tasks want ~cores threads. I/O-bound tasks want more (they idle waiting). Little's Law: threads = required throughput × avg latency.

Bounded queues: reject or block on full. Unbounded queues eventually OOM.

Rejection policy: throw exception, drop, run on caller thread. Choose based on backpressure needs.