Topic Overview

Caching Strategies

Learn different caching patterns and when to use them to improve application performance and reduce database load.

Caching is one of the most effective ways to improve application performance. Understanding different caching patterns helps you choose the right strategy for your use case.

Why Caching?

Caching stores frequently accessed data in fast storage (memory) to avoid expensive operations:

Database queries: Reduce load on database
API calls: Cache external API responses
Computed results: Store expensive calculations
Session data: Fast user session lookups

Trade-offs: Memory vs. disk, consistency vs. performance, complexity vs. simplicity

Cache-Aside (Lazy Loading)

Application checks cache first, loads from database if miss, then stores in cache.

Flow

1. App checks cache for key
2. If hit → return cached value
3. If miss → query database
4. Store result in cache
5. Return value

Implementation

class CacheAsideService {
  constructor(
    private cache: Cache,
    private database: Database
  ) {}

  async get(key: string): Promise<Value> {
    // Try cache first
    const cached = await this.cache.get(key);
    if (cached) {
      return cached;
    }

    // Cache miss - load from database
    const value = await this.database.get(key);
    
    // Store in cache for next time
    await this.cache.set(key, value, { ttl: 3600 });
    
    return value;
  }
}

Pros

Simple to implement
Cache failures don't break application
Only caches data that's actually accessed

Cons

Cache miss penalty (two round trips)
Stale data possible if cache not invalidated
Race conditions possible on concurrent writes

Write-Through

Write to both cache and database simultaneously.

Flow

1. Write to cache
2. Write to database
3. Return success

Implementation

class WriteThroughService {
  async set(key: string, value: Value): Promise<void> {
    // Write to both simultaneously
    await Promise.all([
      this.cache.set(key, value),
      this.database.set(key, value)
    ]);
  }
}

Pros

Cache and database always consistent
No stale data
Read always hits cache (after write)

Cons

Higher write latency (waits for both)
Writes unnecessary data to cache (may never be read)
More complex failure handling

Write-Behind (Write-Back)

Write to cache immediately, write to database asynchronously.

Flow

1. Write to cache
2. Return success immediately
3. Asynchronously write to database (queue/batch)

Implementation

class WriteBehindService {
  private writeQueue: WriteTask[] = [];

  async set(key: string, value: Value): Promise<void> {
    // Write to cache immediately
    await this.cache.set(key, value);
    
    // Queue database write
    this.writeQueue.push({ key, value });
    
    // Process queue asynchronously
    this.processWriteQueue();
  }

  private async processWriteQueue(): Promise<void> {
    while (this.writeQueue.length > 0) {
      const batch = this.writeQueue.splice(0, 100);
      await Promise.all(
        batch.map(task => this.database.set(task.key, task.value))
      );
    }
  }
}

Pros

Very low write latency
Can batch database writes
High write throughput

Cons

Risk of data loss if cache fails before DB write
More complex (need queue, retry logic)
Potential inconsistency window

Refresh-Ahead

Proactively refresh cache before expiration.

Flow

1. Check cache expiration
2. If expiring soon → refresh in background
3. Return current cached value (even if stale)
4. Background refresh updates cache

Use Case

Data that must always be available
Predictable access patterns
Can tolerate slightly stale data briefly

Cache Invalidation Strategies

TTL (Time-To-Live)

Cache expires after fixed time.

await cache.set(key, value, { ttl: 3600 }); // 1 hour

Pros: Simple, automatic cleanup Cons: May serve stale data, unnecessary refreshes

Event-Based Invalidation

Invalidate cache when data changes.

// On database update
await database.update(key, newValue);
await cache.delete(key); // Invalidate cache

Pros: Always fresh data Cons: More complex, need to track dependencies

Version-Based

Include version in cache key, update version on change.

const version = await getVersion(key);
const cacheKey = `${key}:v${version}`;
await cache.get(cacheKey);

Multi-Level Caching

Use multiple cache layers with different characteristics.

L1: In-memory (application) - fastest, smallest
L2: Distributed cache (Redis) - fast, shared
L3: Database - slowest, persistent

Implementation

class MultiLevelCache {
  async get(key: string): Promise<Value> {
    // L1: Local memory cache
    const l1 = this.localCache.get(key);
    if (l1) return l1;

    // L2: Distributed cache (Redis)
    const l2 = await this.redis.get(key);
    if (l2) {
      this.localCache.set(key, l2); // Populate L1
      return l2;
    }

    // L3: Database
    const l3 = await this.database.get(key);
    await this.redis.set(key, l3); // Populate L2
    this.localCache.set(key, l3); // Populate L1
    return l3;
  }
}

Examples

Redis Cache-Aside Pattern

import Redis from 'ioredis';

class UserService {
  constructor(
    private redis: Redis,
    private db: Database
  ) {}

  async getUser(userId: string): Promise<User> {
    const cacheKey = `user:${userId}`;
    
    // Try cache
    const cached = await this.redis.get(cacheKey);
    if (cached) {
      return JSON.parse(cached);
    }

    // Cache miss - load from DB
    const user = await this.db.query(
      'SELECT * FROM users WHERE id = $1',
      [userId]
    );

    // Store in cache (1 hour TTL)
    await this.redis.setex(
      cacheKey,
      3600,
      JSON.stringify(user)
    );

    return user;
  }

  async updateUser(userId: string, updates: Partial<User>): Promise<void> {
    // Update database
    await this.db.update('users', updates, { id: userId });

    // Invalidate cache
    await this.redis.del(`user:${userId}`);
    
    // Or update cache (write-through)
    // const updated = await this.db.get(userId);
    // await this.redis.setex(`user:${userId}`, 3600, JSON.stringify(updated));
  }
}

Cache Warming Strategy

class CacheWarmer {
  async warmCache(): Promise<void> {
    // Pre-load frequently accessed data
    const popularItems = await this.db.query(
      'SELECT * FROM items ORDER BY views DESC LIMIT 1000'
    );

    await Promise.all(
      popularItems.map(item =>
        this.cache.set(`item:${item.id}`, item, { ttl: 3600 })
      )
    );
  }
}

// Warm cache on application startup
app.on('startup', async () => {
  await cacheWarmer.warmCache();
});

Common Pitfalls

Cache stampede: Many requests miss cache simultaneously, all hit database. Fix: Use locks or probabilistic early expiration
Stale data: Not invalidating cache on updates. Fix: Event-based invalidation or shorter TTL
Memory exhaustion: Caching too much data. Fix: Set memory limits, use LRU eviction, cache only hot data
Inconsistent keys: Different services use different cache keys for same data. Fix: Standardize key format
No cache warming: Cold cache on startup causes thundering herd. Fix: Warm cache on startup
Ignoring cache failures: Application breaks if cache is down. Fix: Graceful degradation, fallback to database
Wrong TTL: Too short (frequent misses) or too long (stale data). Fix: Monitor hit rates, adjust based on data change frequency
Not considering cache size: Large objects consume memory quickly. Fix: Compress data, cache only essential fields

Interview Questions

Beginner

Q: What is caching and what are the main caching strategies?

A: Caching stores frequently accessed data in fast storage (usually memory) to avoid expensive operations like database queries.

Main strategies:

Cache-Aside (Lazy Loading): App checks cache, loads from DB on miss, stores in cache
Write-Through: Write to both cache and DB simultaneously
Write-Behind (Write-Back): Write to cache immediately, write to DB asynchronously
Refresh-Ahead: Proactively refresh cache before expiration

Cache-Aside is most common because it's simple and cache failures don't break the app.

Intermediate

Q: Design a caching layer for an e-commerce product catalog that handles 100K reads/second. How do you handle cache invalidation when products are updated?

Architecture:

Multi-level caching: L1 (local), L2 (Redis cluster), L3 (database)
Cache-Aside pattern for reads
Event-based invalidation for updates

Read Flow:

Request → L1 Cache → L2 Cache (Redis) → Database
         (hit)      (hit)              (miss)

Implementation:

class ProductService {
  async getProduct(id: string): Promise<Product> {
    // L1: Local cache (100ms TTL)
    const l1 = this.localCache.get(`product:${id}`);
    if (l1) return l1;

    // L2: Redis (1 hour TTL)
    const l2 = await this.redis.get(`product:${id}`);
    if (l2) {
      this.localCache.set(`product:${id}`, l2, 100);
      return l2;
    }

    // L3: Database
    const product = await this.db.getProduct(id);
    await this.redis.setex(`product:${id}`, 3600, JSON.stringify(product));
    this.localCache.set(`product:${id}`, product, 100);
    return product;
  }
}

Cache Invalidation:

// On product update
async updateProduct(id: string, updates: Partial<Product>): Promise<void> {
  // Update database
  await this.db.updateProduct(id, updates);

  // Invalidate all cache levels
  this.localCache.delete(`product:${id}`);
  await this.redis.del(`product:${id}`);
  
  // Also invalidate related caches
  await this.redis.del(`products:category:*`); // Category listings
  await this.redis.del(`products:search:*`); // Search results
}

// Or use pub/sub for distributed invalidation
async updateProduct(id: string, updates: Partial<Product>): Promise<void> {
  await this.db.updateProduct(id, updates);
  await this.redis.publish('cache:invalidate', `product:${id}`);
}

// All instances subscribe and invalidate local cache
redis.subscribe('cache:invalidate', (key) => {
  this.localCache.delete(key);
});

Scaling:

Redis cluster with sharding (hash by product ID)
Local cache per application instance (reduces Redis load)
Cache warming for popular products on startup
Monitor hit rates (target >90% for L1+L2)

Senior

Q: Design a distributed caching system that maintains consistency across multiple data centers. Products are updated in one region but read from all regions. How do you handle cache invalidation, replication lag, and failover?

Architecture:

Regional Redis clusters in each data center (US, EU, Asia)
Global cache invalidation service using pub/sub
Event sourcing for cache updates
Version vectors for conflict resolution

Multi-Region Cache Strategy:

class GlobalCacheService {
  // Regional caches
  private regionalCaches: Map<Region, Redis> = new Map();
  
  // Global invalidation bus
  private invalidationBus: MessageBus;

  async getProduct(region: Region, productId: string): Promise<Product> {
    // Try regional cache first
    const cached = await this.regionalCaches.get(region)?.get(`product:${productId}`);
    if (cached) {
      return this.validateVersion(cached);
    }

    // Fallback to database
    const product = await this.database.getProduct(productId);
    await this.setProduct(region, productId, product);
    return product;
  }

  async updateProduct(region: Region, productId: string, updates: Partial<Product>): Promise<void> {
    // Update database (source of truth)
    const updated = await this.database.updateProduct(productId, updates);
    const version = updated.version; // Increment version

    // Invalidate all regional caches via global bus
    await this.invalidationBus.publish('product:updated', {
      productId,
      version,
      region, // Origin region
      timestamp: Date.now()
    });

    // Update local region cache immediately (write-through)
    await this.regionalCaches.get(region)?.setex(
      `product:${productId}`,
      3600,
      JSON.stringify({ ...updated, version })
    );
  }
}

Global Invalidation Flow:

Region A: Product updated
  → Database updated
  → Local cache updated (write-through)
  → Publish to global invalidation bus
  → All regions receive invalidation event
  → Regional caches invalidated
  → Next read loads fresh data from DB

Handling Replication Lag:

class VersionedCache {
  async getWithVersion(key: string): Promise<{ value: any, version: number }> {
    const cached = await this.cache.get(key);
    if (!cached) return null;

    // Check if version is stale
    const dbVersion = await this.database.getVersion(key);
    if (cached.version < dbVersion) {
      // Stale - invalidate and reload
      await this.cache.delete(key);
      return null;
    }

    return cached;
  }
}

Failover Strategy:

Primary region fails: Route reads to secondary region cache
Cache miss in secondary: Load from database (cross-region read)
Warm secondary cache: Replicate popular items to secondary
Circuit breaker: If secondary latency > threshold, fail fast

Consistency Levels:

Strong consistency: Always read from database (higher latency)
Eventual consistency: Read from cache, accept brief staleness (lower latency)
Session consistency: User always sees their own writes immediately

Monitoring:

Cache hit rates per region
Invalidation latency (time to propagate)
Staleness metrics (version mismatches)
Cross-region read latency

Optimization:

Predictive invalidation: Invalidate related caches (category, search) proactively
Batch invalidation: Group invalidations to reduce message bus load
Regional warming: Pre-populate secondary regions with hot data
TTL-based fallback: Even with invalidation, use TTL as safety net

Key Takeaways

Cache-Aside is most common: simple, resilient to cache failures
Write-Through ensures consistency but higher write latency
Write-Behind maximizes write performance but risks data loss
Multi-level caching (L1 local, L2 distributed, L3 database) optimizes for different access patterns
Cache invalidation is critical: use TTL, event-based, or version-based strategies
Cache stampede prevention: Use locks or probabilistic early expiration
Monitor hit rates: Target >80-90% for effective caching
Graceful degradation: Application should work even if cache fails
Global caching: Use pub/sub for cross-region invalidation, version vectors for consistency
Cache warming: Pre-load popular data on startup to avoid cold cache