Rootconf Mini 2024

TLDR;

An advanced exploration of war stories from building caching systems at a decacorn.

Description

There is a reason why the following quote is famous:

“There are only two hard things in Computer Science: cache invalidation and naming things.” — Phil Karlton

Caching is everywhere; it could be a simple system of storing only the most popular books in a shop and keeping all other books in the backroom, or the sophisticated ways we build caching in software systems. This ranges from a simple cache layer on top of a database to using complex data structures like bloom filters as a cache for more expensive checks to the storage layer.

Topics Covered

Ways & Wins

Exploring various forms of caching systems, characteristics of a caching system and numerous innovative ways a caching system gets deployed at scale.

Examples: Multiple cache layers, efficiency gains, eviction policies, metrics, consistency, Saving on latency, saving on cost, serving as a fallback, predictable response times, improved resiliency.

Woes & Weird

Despite the benefits, caching systems can introduce complexities and challenges that require careful consideration. They also surface unusual and unexpected behaviors that emerge when working with caching systems at scale in terms of use cases and scale.

Examples: Cache Stampede, Bi-Modal Behavior, Timeout Propagation Issues, Cache Poisoning, Key Collisions, Caching Increasing Response Time, Infinite Loops on Cache Invalidation, Negative Caching, Self-Immolating Caches, Cache Misses Triggering Batch Jobs

Wisdom

Best practices and lessons learned from building and maintaining caching systems.

Examples: Effective Key Management, Monitoring and Profiling, Handling Cache Errors Gracefully, Avoiding Misuse of Caching Systems, Data Serialization Strategies

Target Audience & Prerequisites

The talk targets slightly experienced folks (1–2+ years of experience in backend) or even those who are early in their career if they have spent some time in systems thinking.

In terms of prerequisites, AWS has built a set of pretty good resources around caching: https://aws.amazon.com/caching/. The Systems Design Roadmap also provides a path to start more research around caching: https://roadmap.sh/system-design.

For those interested in a more advanced perspective, this paper is a very good entry point around patterns and behaviors observed when managing large-scale caching systems (both in terms of infrastructure and various use cases): https://www.usenix.org/conference/osdi20/presentation/yang

Slide Deck

Draft: https://docs.google.com/presentation/d/1L9oHgnhioVKZikgdf-v5918PXqWhk9uCF2Pg9QejVac/edit?usp=sharing