Cache Stampede: Causes, Prevention, and Best Practices

In my previous two articles "Getting Started with Caching in .NET" and "A Deep Dive into Cache Eviction Policies" , I explained caching and cache eviction. A cache provides runtime storage for frequently accessed data, while cache eviction is the process of removing data from the cache.

A cache stampede is a situation that occurs when data is removed from the cache due to expiration or eviction, and multiple requests simultaneously try to fetch the same data from the underlying data source. In this article, we will discuss cache stampedes and explore different ways to prevent them.

What is Cache Stampede

A cache stampede (also called the thundering herd problem or dogpile effect) occurs when many requests simultaneously try to fetch the same data after a cache entry expires or is missing. Instead of one request regenerating the data and updating the cache, all requests hit the backend at once, potentially overwhelming the database or service.

Example
Imagine your application caches a popular product page for 5 minutes.

1. Thousands of users request the page.
2. The cache serves the data quickly.
3. The cache entry expires.
4. At that exact moment, 1,000 new requests arrive.
5. Since the cache is empty, all 1,000 requests query the database.
6. The database experiences a sudden spike in load, increasing latency or even failing.

Data Flow Example

Before expiration:
			 Users (1000)
				  ↓
			 Cache (HIT)
				  ↓
			 Response
After expiration:
			Users (1000)
				 ↓             
              Key Expired"
    			 ↓
			 1000 cache misses
              	 ↓
			 1000 DB queries for the SAME data
    			 ↓
			   DB overloaded
    			 ↓
             DB becomes slow/timeouts
    			 ↓
			 More retries
    			 ↓
			System can collapse

Cache Stampede Causes

1. Hot key expires
   • A highly requested key has a TTL.
   • It expires.
   • Thousands of requests miss it simultaneously.
   • Every request hits the DB/API to rebuild the same value.
2. Synchronized TTLs
   • Many keys are created around the same time with the same TTL.
   • They expire together, causing a large wave of cache misses—sometimes called a cache avalanche.
3. No request coalescing / locking
   • Nothing ensures that only one request regenerates a missing value.
   • So N concurrent requests produce N identical DB queries.
4. Cold cache
   • Cache restart, deployment, flush, or eviction can remove many frequently used entries.
   • Traffic immediately falls through to the backend.
5. Slow cache regeneration
   • If rebuilding the value takes a long time, more requests arrive while the cache is still empty.
   • This increases the number of concurrent backend requests.
6. Retry amplification
   • The DB/API becomes slow because of the stampede.
   • Clients retry timed-out requests.
   • Those retries create even more load, making recovery harder.

Common Solutions

1. Request coalescing (Single Flight): Allow only one request to regenerate the cache.
2. Distributed locking

   If you have multiple application servers, use a distributed lock (for example, using Redis) so only one server rebuilds the cache.

   
   Server A ---- acquires Redis lock ----> DB
   Server B ---- waits
   Server C ---- waits
   

3. Stale-While-Revalidate (SWR)
   Instead of removing expired data immediately:
   • Serve the slightly stale cached value.
   • Refresh it in the background.

   
     User
       │
       ▼
     Cache (stale)
       │
       ├──► Return stale response immediately
       │
       └──► Background refresh
                 │
                 ▼
              Database
   
   

   This minimizes user-facing latency.
4. Randomized expiration (TTL jitter)
   If many keys are created at the same time with the same TTL, they may all expire together.
   
   Instead of: TTL = 300 seconds
   Use: TTL = 300 + random(0-60)
   
   This spreads expirations over time and reduces synchronized load spikes.
5. Cache warming (preloading) Refresh frequently accessed cache entries before they expire.
6. Never let hot keys expire
   For extremely popular data:
   • Use a very long TTL or no expiration.
   • Invalidate the cache explicitly when the underlying data changes.

Important: In modern high-traffic systems, it's common to combine several techniques for example, request coalescing, stale-while-revalidate, and TTL jitter to keep backend load stable even when cache entries expire.

Best Practices

The best practice is to combine several defenses, rather than rely on one technique. A cache stampede (thundering herd) happens when a popular key expires and many requests simultaneously hit the database/backend.

Summary

A practical guide to cache stampedes, covering how simultaneous cache misses can overwhelm backend services, common causes, real-world scenarios, and proven prevention techniques.

Thanks