Caching Strategies

A cache is a small, fast copy of data kept close to where it’s needed, so you don’t have to recompute or re-fetch it from the slow source every time. It is the single highest-leverage latency move in systems engineering: a well-placed cache can turn a 50 ms database read into a 0.5 ms memory read, and it can absorb the read load that would otherwise crush your database. But every cache introduces the same deep problem — keeping the copy in step with the source — and how you wire the cache to the source defines the five patterns below. (For what a cache is and the eviction mechanics, see Caching.)

The question to hold throughout: what does this buy us, and what does it cost? Caching always buys speed and offloads the source. It always costs you a second copy of the truth that can drift.

The read patterns

Cache-aside (lazy loading)

The application is in charge. On a read it checks the cache; on a miss it reads the source, then puts the value into the cache itself before returning.

   read: app ─► cache?  ─hit─► return
                  │miss
                  ▼
               app ─► DB ─► (app writes value into cache) ─► return

What does it buy us, and what does it cost? It buys simplicity and resilience — the cache is just a side store; if it’s down, you still serve from the DB. Only data that’s actually requested gets cached, so you don’t waste memory. It costs you a cold-cache penalty (the first request for each key always misses) and the risk of stale data, because the app, not the cache, owns freshness. This is by far the most common pattern in the wild.

Read-through

Same read flow, but the cache library handles the miss: the app only ever talks to the cache, and the cache fetches from the source on a miss and stores it.

What does it buy us, and what does it cost? It buys cleaner application code — caching logic lives in one place, not scattered across every call site. It costs you a tighter coupling to a cache that knows how to load from your source, and the same cold-start misses as cache-aside.

The write patterns

How you handle writes is where the patterns really diverge, and it’s where consistency lives.

Write-through

Every write goes to the cache and the source synchronously, before the write is acknowledged.

   write: app ─► cache ─► DB ─► ack   (both updated before returning)

What does it buy us, and what does it cost? It buys strong cache freshness — the cache is never behind the database, so reads right after a write are correct. It costs you write latency (every write pays for two stores) and wasted caching of data that may never be read again.

Write-back (write-behind)

Write to the cache, acknowledge immediately, and flush to the source asynchronously later (batched, coalesced).

   write: app ─► cache ─► ack
                   │  (later, async)
                   ▼
                  DB

What does it buy us, and what does it cost? It buys blazing write throughput and low latency — the slow source is off the critical path, and many writes to the same key collapse into one DB write. It costs you durability risk: if the cache dies before flushing, those writes are gone. This is the most powerful and the most dangerous write pattern; use it only where some loss is tolerable or the cache is itself durable/replicated.

Write-around

Writes go straight to the source, skipping the cache; the cache is only populated later on a read (via cache-aside/read-through).

What does it buy us, and what does it cost? It buys protection against cache pollution — data that’s written but rarely read never displaces hot data in the cache. It costs you a guaranteed miss on the first read after a write (the freshly written value isn’t cached yet). Great for write-heavy, read-rarely data like logs.

Pairing reads and writes

Read and write patterns combine. The classic, safe default is cache-aside reads + write-through or write-around writes. Reach for write-back only when write throughput is the binding constraint and you’ve made peace with the durability risk.

The hard part: invalidation

“There are only two hard things in Computer Science: cache invalidation and naming things.” — Phil Karlton

Every pattern above shares one unavoidable problem: the cached copy can become stale when the source changes. There are three families of answers, and each trades freshness against effort:

• TTL (expire after N seconds). Simple and self-healing — staleness is bounded by the TTL. The cost is you serve data up to N seconds old, and you re-fetch on a schedule whether the data changed or not. The default for most caches.
• Explicit invalidation (delete/update the key on write). Tightest freshness, but you have to find every cache entry a write affects — easy for one key, brutal for derived/aggregated data. Miss one and you serve stale data forever.
• Write-through/write-back. The write path keeps the cache fresh by construction, sidestepping the “who deletes the key” problem for the keys it touches.

The two stale-data traps

Over-caching serves wrong answers (you forgot to invalidate). Under-caching / over-invalidating serves slow answers and stampedes (you blow away entries too eagerly and every miss hits the DB at once). The sweet spot is narrow — and when many clients miss the same hot key simultaneously you get a cache stampede / thundering herd, a failure mode worth understanding before you rely on caching in production.

A quick comparison

Pattern	Who loads on miss	Write freshness	Main cost
Cache-aside	App	App’s job (stale)	Cold misses, stale risk
Read-through	Cache	App’s job (stale)	Coupling to cache loader
Write-through	—	Always fresh	Slow writes
Write-back	—	Fresh, eventually	Durability risk
Write-around	App (on read)	Fresh in DB only	First-read miss

Check your understanding

1. In cache-aside, who is responsible for putting a value into the cache after a miss, and why does that make the cache resilient to its own failure?
2. Contrast write-through and write-back in terms of latency and durability risk.
3. When would you deliberately choose write-around, and what does it protect against?
4. Why is explicit invalidation “tighter” than TTL but much harder to get right?
5. Explain how over-invalidating can be as harmful as forgetting to invalidate.