Where caches live
A cache keeps a copy of data somewhere faster or closer than its source. It works when reads repeat and slightly stale data is acceptable. There are caches at every layer:
- Browser / client: HTTP cache headers (
Cache-Control,ETag). - CDN: static files and cacheable pages at edge locations near users.
- Application cache: Redis or Memcached in front of the database, shared by app servers.
- In-process cache: a map inside each server; fastest, but each server has its own copy.
- Database: its own buffer pool and query caches.
Read and write strategies
| Strategy | How it works | Trade-off |
|---|---|---|
| Cache-aside (lazy loading) | App reads the cache; on a miss, reads the DB and fills the cache | Most common. First read is slow; data can go stale until expiry or invalidation |
| Read-through | The cache itself loads from the DB on a miss | Simpler app code; needs a cache that supports it |
| Write-through | Writes go to the cache and the DB together | Cache always fresh; every write pays both costs |
| Write-back (write-behind) | Writes go to the cache; flushed to the DB later | Fast writes; data loss if the cache dies before flushing |
| Write-around | Writes go to the DB only; cache fills on reads | Avoids caching data that's never read again |
// cache-asideUser getUser(long id) {User u = cache.get("user:" + id);if (u != null) return u; // hitu = db.findUser(id); // miss: go to the sourcecache.set("user:" + id, u, Duration.ofMinutes(10));return u;}void updateUser(User u) {db.save(u);cache.delete("user:" + u.id()); // invalidate, don't update}
On writes, deleting the cache entry is usually safer than updating it: two concurrent updates can otherwise leave the cache holding the older value.
Eviction and expiry
A cache is smaller than its source, so it must choose what to drop. LRU (least recently used) evicts the entry untouched for longest and is the usual default; LFU keeps frequently used items; FIFO is simplest. A TTL (time to live) bounds how stale an entry can get, even if nobody invalidates it.
Classic caching problems
- Stampede (thundering herd): a hot key expires and thousands of requests hit the DB at once. Fixes: one request recomputes while others wait (a lock or request coalescing), refresh early before expiry, add jitter to TTLs.
- Penetration: requests for keys that don't exist always miss. Fixes: cache the "not found" result briefly, or check a Bloom filter first.
- Hot keys: one key gets so much traffic that its cache node melts. Fixes: replicate the key, add a small in-process cache, or split it into several keys.
- Consistency: the cache and DB disagree. Accept bounded staleness with TTLs, invalidate on write, or use change-data-capture to invalidate from the DB's log.
Cache-aside read path
GET /users/42: the app checks the cache for user:42. It's a miss.
See it animated
Interview questions
Try answering out loud first, then open the model answer.
Q1Explain cache-aside. What are its downsides?intermediate
The app checks the cache, and on a miss reads the database and populates the cache. It's simple and only caches what's read. Downsides: every miss costs three trips, data can be stale until TTL or invalidation, and a cold cache after a restart sends a burst of load to the database.
Q2Write-through vs write-back?intermediate
Write-through writes to the cache and the database synchronously, so the cache is never stale but writes are slower. Write-back acknowledges after writing the cache and persists later: much faster writes and natural batching, but data is lost if the cache fails before flushing.
Q3What is a cache stampede and how do you prevent it?advanced
When a popular entry expires, many requests miss together and all hit the database. Prevent it by letting only one request rebuild the value (a mutex or single-flight), serving the stale value while refreshing in the background, refreshing probabilistically before expiry, and adding random jitter to TTLs so keys don't expire together.
Q4How would you implement an LRU cache?intermediate
A hash map from key to node plus a doubly linked list ordered by recency. get moves the node to the front; put inserts at the front and, when over capacity, removes the tail. Both operations are O(1). In Java, LinkedHashMap with access order and removeEldestEntry does this.
Q5When should you not cache?intermediate
When data must always be fresh (account balances during a transfer), when reads rarely repeat, when writes far outnumber reads, or when the source is already fast enough. A cache adds a consistency problem and an extra component to run.
Quick check
0 of 4 answered