Why architecture matters here

Cache stampedes take down origins. Architecture matters because each layer addresses a different attack vector.

Advertisement

The architecture: every piece explained

The top strip is the primary. Client request. Cache miss. Singleflight. Origin fetch.

The middle row is preventive. Probabilistic early. TTL jitter. Async refresh. Circuit breaker.

The lower rows are ops. Locking pattern. Metrics. Ops — capacity + tuning + drills.

Cache stampede — singleflight + probabilistic early expiration + jitter + async refreshprotect origin from cold-cache thundering herdClient requestcache lookupCache misshot key expiredSingleflightcoalesce concurrentOrigin fetchone call for manyProbabilistic earlyrefresh before expiryTTL jitterspread expirationsAsync refreshserve stale + refreshCircuit breakerprotect originLocking patternavoid overlapMetricsstampede countOps — capacity + tuning + drillsrefreshjitterserve staleguardcoordinatewatchwatchoperateoperate
Cache stampede prevention layers.
Advertisement

End-to-end flow

End-to-end: hot cache entry near expiry. Probabilistic early refresh fires in one worker; async refresh brings new value; others get stale + the old TTL. Singleflight coalesces stampede if it happens. Circuit breaker if origin fails.