Caching — System Design

Caching is the art of remembering answers. Instead of computing or fetching data every time, you store the result close to the consumer. A cache hit is instant — memory speed. A cache miss goes all the way to disk. The difference is often 100x to 1000x.

Three-Layer Cache Stack

Browser cache (top) ? Server cache (middle) ? Database (bottom). Watch a Cache Hit stop at the top layer vs a Cache Miss fall all the way through.

drag — scroll to zoom

Code example

A simple in-memory cache with TTL, and Redis commands for production.

CACHE IMPLEMENTATION

// Simple in-memory cache with TTL
class Cache {
    constructor() { this.store = new Map(); }

    get(key) {
        const entry = this.store.get(key);
        if (!entry) return null; // cache miss
        if (Date.now() > entry.exp) {
            this.store.delete(key); return null; // expired
        }
        return entry.value; // cache hit ?
    }

    set(key, value, ttlMs = 60_000) {
        this.store.set(key, { value, exp: Date.now() + ttlMs });
    }
}

const cache = new Cache();

async function getUser(id) {
    const cached = cache.get(`user:${id}`);
    if (cached) return cached; // ? cache hit, fast!

    const user = await db.findUser(id); // ? database
    cache.set(`user:${id}`, user, 5 * 60_000); // TTL 5min
    return user;
}
// Redis — production-grade caching
const redis = require('redis');
const client = redis.createClient();
await client.connect();

// SET with TTL (seconds)
await client.set('user:42', JSON.stringify(user), {
    EX: 300   // expire in 5 minutes
});

// GET — returns null on miss
const raw = await client.get('user:42');
if (raw) {
    return JSON.parse(raw); // ? cache hit, microseconds
}

// Atomic increment (counters, rate limiting)
await client.incr('pageviews:home');

// LRU eviction — Redis evicts least-recently-used
// when memory is full. Set in redis.conf:
// maxmemory-policy allkeys-lru
      

Key concepts

⚡

Cache Hit

instant — memory — green path

The requested data is in the cache. Returns immediately — memory speed, microseconds. No database trip. The goal of every caching strategy.

❌

Cache Miss

slow — fallthrough — red path

Data not in cache. Falls through to the database. Result is stored in the cache on return so the next request is a hit. Miss on first access is normal.

🗑

Eviction Policy

LRU — TTL — LFU

What gets removed when cache is full? LRU (Least Recently Used) evicts oldest. TTL (Time To Live) expires entries after a duration. LFU evicts least-used.

🔴

Redis

in-memory — production — fast

The industry-standard cache. Sub-millisecond reads. Supports strings, hashes, lists, sets, sorted sets, pub/sub, and atomic operations. Used by Twitter, GitHub, StackOverflow.

→ Databases System Design — Algo3DVerse Load Balancing ?