Search
⌘K
Full Article
Quick Reference

Caching

When to Use Caching

1

Read-heavy workloads

The same data is accessed frequently.

2

Expensive computations or queries

Results can be stored and reused.

3

Data that doesn't change often

Or where slight staleness is acceptable.

4

Reducing load on downstream services

Take pressure off databases and APIs.

Cache Architectures

The four cache architectures: Cache-Aside (app reads cache, on miss reads database then populates), Write-Through (write to cache and database at the same time), Write-Behind (write to cache first, later flush to database), and Read-Through (cache fetches from database on miss for the app)
  • Cache-Aside (Lazy Loading): App checks cache first, on miss reads from DB and populates cache. Most common pattern.
  • Write-Through: Every write goes to cache AND DB simultaneously. Ensures consistency but adds write latency.
  • Write-Behind (Write-Back): Writes go to cache first, async flush to DB. Fast writes but risk of data loss.
  • Read-Through: Cache itself handles fetching from DB on miss. Simplifies app logic.

Eviction Policies

The four eviction policies: LRU evicts the least recently accessed item (most common default), LFU evicts the least frequently accessed item (good for skew), TTL expires items after a set duration (simple, predictable), and FIFO is first in first out (rarely used in practice)
  • LRU (Least Recently Used): Evicts the least recently accessed item. Most common default.
  • LFU (Least Frequently Used): Evicts the least frequently accessed item. Good for skewed access patterns.
  • TTL (Time-To-Live): Items expire after a set duration. Simple and predictable.
  • FIFO: First in, first out. Rarely used in practice.

Cache Types

The four cache types: In-Process Cache lives in app memory (fastest, not shared across instances), External Cache is shared across instances and adds a network hop (~1ms), CDN Cache has edge servers caching static assets close to users, and Client-Side Cache is the browser cache using HTTP headers like Cache-Control and ETag
  • In-Process Cache: Lives in app memory (HashMap). Fastest but not shared across instances.
  • External Cache (Redis/Memcached): Shared across instances. Adds network hop (~1ms).
  • CDN Cache: Edge servers cache static assets close to users. Great for images, JS, CSS.
  • Client-Side Cache: Browser cache, HTTP cache headers (Cache-Control, ETag, Last-Modified).

Key Numbers

The key caching numbers compared as bars, fastest to slowest: an in-process cache responds in nanoseconds, Redis handles 100k+ ops/sec with sub-millisecond latency, Memcached has similar throughput but is simpler with no persistence, a CDN hit takes ~5-20ms, and origin ~100-500ms
Redis

100k+ ops/sec, sub-millisecond latency

Memcached

Similar throughput, simpler (no persistence)

CDN hit

~5-20ms vs origin ~100-500ms

In-process cache

Nanoseconds

Common Pitfalls

The four common caching pitfalls: Cache Stampede (many requests miss at once, overwhelming the backend), Stale Data (cache serves outdated values instead of fresh data), Cold Start (empty cache after restart overloads the database), and Over-caching (caching everything wastes memory, cache only hot data)
  • Cache Stampede: Many requests hit cache miss simultaneously. Fix: locking, request coalescing.
  • Stale Data: Cache returns outdated data. Fix: TTL, cache invalidation on writes.
  • Cold Start: Empty cache after restart causes DB overload. Fix: cache warming.
  • Over-caching: Caching everything wastes memory. Cache only hot data.

Your account is free and you can post anonymously if you choose.

Reading Progress

On This Page

When to Use Caching

Cache Architectures

Eviction Policies

Cache Types

Key Numbers

Common Pitfalls

Questions
Meta SWE Interview QuestionsAmazon SWE Interview QuestionsGoogle SWE Interview QuestionsOpenAI SWE Interview QuestionsEngineering Manager (EM) Interview Questions
Learn
Learn System DesignLearn DSALearn BehavioralLearn ML System DesignLearn Low Level DesignGuided Practice
Links
FAQPricingGift PremiumHello Interview Premium
Legal
Terms and ConditionsPrivacy PolicySecurity
Contact
About UsProduct Support

7511 Greenwood Ave North Unit #4238 Seattle WA 98103

© 2026 Optick Labs Inc. All rights reserved.