Why architecture matters here
LRU fails on specific workloads. Sequential scans evict hot data. Zipfian workloads benefit more from LFU or W-TinyLFU. Naive locking makes the cache slower than not caching. Weightless capacity handles fixed-size entries but not variable ones.
The architecture matters because tuning + variant choice depend on your access pattern. Concurrency depends on your hit rate and thread count.
With the pieces mapped, you can build a cache that actually helps.
The architecture: every piece explained
The top strip is the core algorithm. Key lookup probes a hash map for O(1) find. Linked list maintains recency; on access, move the node to the head. Head / tail are the MRU and LRU endpoints. Eviction on full drops the tail.
The middle row is production concerns. Admission policy (TinyLFU) decides whether new candidates deserve to displace existing entries. Concurrency uses sharding by hash or lock-free structures. TTL + expiry handles time-based eviction alongside LRU. Weighted entries accommodate variable-sized items with size-aware capacity.
The lower rows are ops. Metrics track hit rate, eviction rate, and size. Variants — LFU, ARC, W-TinyLFU — handle different access patterns. Ops covers cache sizing, shard fan-out, and persistence.
End-to-end flow
End-to-end: an app cache uses W-TinyLFU with sharded concurrency. Request arrives; hash key + shard select; hash map probe; hit → return + move to MRU. Miss → fetch from DB; admission policy compares to a TinyLFU sketch to decide keeping; if kept, insert at head; if cache full, drop tail; if candidate loses, don't insert. Metrics show hit rate 82%, eviction rate steady. Under sequential scan, admission policy protects hot entries from being displaced.