Caching 101

When You Need Caching 101

Caching reduces latency (memory access: ~100ns vs disk: ~10ms), reduces database load (cache absorbs 80-95% of reads), and reduces costs (fewer database queries = smaller database needed). Every production system uses caching.

What It Is

Caching stores frequently accessed data in a fast storage layer (in-memory) so future requests can be served without hitting the slower origin (database, API, disk). Caching is the single most effective technique for improving system performance.

How It Works

Application requests data → check Redis cache → if hit, return cached data (1ms). If miss, query database (50ms), store result in Redis with TTL, return data. Subsequent requests for the same data hit the cache.

For write operations, you must decide how to keep the cache and database in sync (see caching strategies).

The Decision Framework

• Cache hit/miss: A hit means data was found in cache. A miss means data must be fetched from origin and cached for future use.
• Cache layers: Browser cache → CDN → Application cache (Redis) → Database query cache → OS page cache.
• Cacheability: Cache data that is read frequently, changes infrequently, and is expensive to compute. Do NOT cache sensitive data or rapidly changing data.
• TTL (Time to Live): Every cache entry should have an expiration time. Prevents serving infinitely stale data.
• Cache warming: Pre-populate the cache before traffic arrives (after deployment, after cache flush) to avoid a thundering herd of cache misses.

What the Industry Uses

Facebook caches social graph data in Memcached — 75 billion requests per day served from cache.

Twitter caches timelines in Redis. When you open Twitter, your feed is served from cache, not computed on the fly.

Stack Overflow caches HTML fragments, query results, and user sessions in Redis, serving millions of requests per second.

Performance and Tradeoffs

• Freshness vs Performance: Longer TTL = faster but potentially stale. Shorter TTL = fresher but more origin hits.
• Memory vs Hit rate: Bigger cache = higher hit rate but more expensive.
• Consistency: Cache and database can get out of sync if invalidation fails.

Mistakes Engineers Make

1. Caching without a TTL — stale data stays forever
2. Not handling cache misses gracefully — thundering herd crashes the database
3. Caching too much — fills memory with rarely accessed data
4. Not monitoring cache hit rate — a low hit rate means caching is not helping

Practice These Interview Questions

1. What is caching and why is it important?
2. What data is suitable for caching?
3. What is cache invalidation and why is it hard?
4. What is a thundering herd problem?

Caching in .NET Applications

The .NET ecosystem provides multiple caching layers, from in-memory to distributed:

IMemoryCache for single-server caching — built into ASP.NET Core, zero configuration:

// Register in Program.cs
builder.Services.AddMemoryCache();

// Use in your service
public class CatalogService
    private readonly IMemoryCache _cache;
    public Product GetProduct(int id)
        return _cache.GetOrCreate($"product:{id}", entry =>
            entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5);
            return _dbContext.Products.Find(id);
        );

IDistributedCache for multi-server caching — backed by Redis in production. This is critical when you have multiple application servers behind a load balancer. Every server must see the same cached data.

Response Caching Middleware — cache entire HTTP responses at the middleware level. Add [ResponseCache(Duration = 60)] to controller actions for static content like product pages or blog posts.

Real example at scale: Microsoft Teams (built on .NET) caches user presence data, channel memberships, and recent messages in Redis. When you open Teams and see "2 unread messages" instantly, that is coming from cache — not a database query.

Further Reading

• caching-strategies
• cache-eviction-policies
• distributed-caching
• cdn

The Real-World Incident That Made This Famous

Understanding Caching 101 became critical after multiple high-profile production incidents at major tech companies. When systems handle millions of users, even small misunderstandings about Caching 101 can lead to cascading failures that cost millions in lost revenue and erode user trust. Companies like Netflix, Google, Amazon, and Meta have all invested heavily in mastering Caching 101 because they learned the hard way that ignoring it leads to outages.

The key lesson from these incidents: Caching 101 is not just a theoretical concept — it is a practical skill that separates engineers who build resilient systems from those who build fragile ones.

How Senior Engineers Think About This

Senior engineers approach Caching 101 differently from textbook definitions. Instead of memorizing rules, they build mental models. They ask: "What problem does Caching 101 solve? When does it fail? What are the alternatives?" This problem-first thinking leads to better design decisions because every system has unique constraints.

When evaluating Caching 101 in a system design context, experienced engineers consider the failure modes first. What happens when this component goes down? How does the system degrade? Is the degradation graceful or catastrophic? These questions reveal more about your understanding than any textbook definition.

Common Interview Mistakes

Mistake 1: Giving a textbook definition without context. Interviewers want to see you connect Caching 101 to real systems and real problems.

Mistake 2: Not discussing trade-offs. Every design decision involving Caching 101 has trade-offs. Discuss what you gain and what you give up.

Mistake 3: Overcomplicating the solution. Start with the simplest approach to Caching 101 that meets the requirements, then add complexity only when justified.

Production Checklist

• Define clear metrics for measuring the effectiveness of your Caching 101 implementation
• Set up monitoring and alerting that specifically tracks Caching 101-related failures
• Document your Caching 101 design decisions in Architecture Decision Records (ADRs)
• Test failure scenarios related to Caching 101 in staging before production deployment
• Review and update your Caching 101 implementation quarterly as system requirements evolve
• Train new team members on the specific Caching 101 patterns used in your system

Read the original source | Content from System-Design-Overview