Why architecture matters here
The buffer pool exists because the executor's unit of work — a tuple, an index entry — is orders of magnitude smaller than the storage layer's economic unit of transfer, the page. Reading a 100-byte row costs an 8KB page read in PostgreSQL and a 16KB read in InnoDB, and workloads revisit the same pages constantly: index roots on every traversal, hot rows on every checkout, catalog pages on every plan. Without a shared cache, every concurrent query would re-read the same B-tree root thousands of times per second. The buffer pool amortizes that cost across the whole server, which is why its hit ratio is the single most predictive number for OLTP latency: at 99% hit ratio a query touching 50 pages does half an I/O on average; at 90% it does five.
But caching is the easy third of the job. The buffer pool is also the concurrency boundary — multiple backends read and modify the same frame, so every page carries a pin count (how many operations are using it right now) and a latch (a short-lived lock protecting the bytes themselves, distinct from transactional row locks). And it is the recovery boundary: the pool holds dirty pages whose in-memory contents are newer than the on-disk copy, so eviction and crash safety are coupled through the write-ahead log. The steal/no-force policy that virtually every serious engine adopts — dirty pages may be written before commit, and need not be written at commit — is only sound because the WAL is forced first. Understanding the buffer pool therefore means understanding the flush-ordering contract, not just the cache. Engines that get this wrong corrupt data; engines that get it right but tune it badly stall every few minutes when the checkpointer panics. Both failure classes trace back to the same diagram.
The architecture: every piece explained
Page table. The entry point is a hash table keyed by page identifier — in PostgreSQL a (tablespace, relation, fork, block number) tuple — mapping to a frame index. It is partitioned into many latch regions (128 in PostgreSQL) so concurrent lookups don't serialize on one lock. A hit returns the frame and increments its pin count atomically; a miss triggers the replacement path.
Frames and buffer descriptors. The pool itself is a contiguous array of page-sized frames plus a parallel array of descriptors: page id, pin count, usage counter, dirty flag, and a content latch. The pin count is the memory-safety mechanism — an evictor may only claim a frame whose pin count is zero. The dirty flag, set when any modification is logged against the page, is the recovery hook.
Replacement policy. Plain LRU dies horribly under sequential scans — one large table scan evicts the entire working set. Every engine ships a scan-resistant variant. PostgreSQL uses clock sweep with a usage counter (capped at 5) and, separately, small ring buffers for bulk reads so scans recycle their own frames. InnoDB uses a midpoint-insertion LRU: new pages enter 5/8 of the way down the list and must survive innodb_old_blocks_time (1 second by default) before promotion to the young half, so a scan's pages age out of the cold region without touching the hot one. LRU-K, used in several commercial engines, tracks the K-th most recent access to distinguish one-hit wonders from genuinely warm pages.
Write-back machinery. Three actors flush dirty pages. The background writer trickles them out ahead of demand so foreground evictions rarely hit a dirty victim. The checkpointer periodically writes everything dirty as of a checkpoint's start (a fuzzy checkpoint — the system keeps running) and then records that recovery may begin at the checkpoint's WAL position, bounding restart time. Finally, any backend that must evict a dirty page flushes it itself — the slow path good tuning avoids. InnoDB adds a doublewrite buffer and PostgreSQL full-page images to defeat torn pages: a 16KB page write is not atomic on 4KB-sector hardware, so both engines arrange to have a full intact copy somewhere before overwriting the home location.
End-to-end flow
Trace a read. A backend needs heap block 4711 of table orders. It hashes the page id, takes the relevant page-table partition latch, and finds a hit: frame 88213. It pins the frame, drops the partition latch, takes the content latch in shared mode, reads the tuples it needs, releases the latch, and unpins. Total cost: two atomic increments and a hash probe — tens of nanoseconds. On a miss the backend instead pulls a frame from the free list (empty after warm-up) or runs the clock hand until it finds an unpinned frame with usage count zero, decrementing counters as it sweeps. If the victim is dirty, the backend must first force the WAL up to that page's most recent log record (the WAL-before-data rule), write the page out, and only then reuse the frame — this is the latency cliff hidden inside a "simple" read.
Now trace a write. An UPDATE modifies a tuple: the backend pins the page, takes the content latch exclusively, generates a WAL record describing the change, stamps the page with that record's log sequence number, applies the bytes in memory, sets the dirty flag, and releases. Nothing is written to the data file. Commit forces the WAL to disk — sequential I/O, group-committed across transactions — and returns. The dirty page lingers in the pool, possibly absorbing dozens more updates (write coalescing is a major buffer pool dividend), until the background writer, the checkpointer, or an eviction finally writes it, checking first that the WAL has been flushed past the page's stamped LSN. After a crash, recovery starts at the last checkpoint, replays WAL forward, and uses each page's LSN to skip records already reflected on disk. The pool, the log, and recovery form one machine: the flush order is the correctness invariant, and everything else is performance.