Rootconf Mini 2024

Geeking out on systems and security since 2012

Tickets

Loading…

Srinivas Devaki

Srinivas Devaki

@srioptiowl

Art of Caching: Ways, Wins, Woes, Weird, Wisdom

Submitted Oct 17, 2024

TLDR;

An advanced exploration of war stories from building caching systems at a decacorn.

Description

There is a reason why the following quote is famous:

“There are only two hard things in Computer Science: cache invalidation and naming things.” — Phil Karlton

Caching is everywhere; it could be a simple system of storing only the most popular books in a shop and keeping all other books in the backroom, or the sophisticated ways we build caching in software systems. This ranges from a simple cache layer on top of a database to using complex data structures like bloom filters as a cache for more expensive checks to the storage layer.

Topics Covered

Ways & Wins

Exploring various forms of caching systems, characteristics of a caching system and numerous innovative ways a caching system gets deployed at scale.

Examples: Multiple cache layers, efficiency gains, eviction policies, metrics, consistency, Saving on latency, saving on cost, serving as a fallback, predictable response times, improved resiliency.

Woes & Weird

Despite the benefits, caching systems can introduce complexities and challenges that require careful consideration. They also surface unusual and unexpected behaviors that emerge when working with caching systems at scale in terms of use cases and scale.

Examples: Cache Stampede, Bi-Modal Behavior, Timeout Propagation Issues, Cache Poisoning, Key Collisions, Caching Increasing Response Time, Infinite Loops on Cache Invalidation, Negative Caching, Self-Immolating Caches, Cache Misses Triggering Batch Jobs

Wisdom

Best practices and lessons learned from building and maintaining caching systems.

Examples: Effective Key Management, Monitoring and Profiling, Handling Cache Errors Gracefully, Avoiding Misuse of Caching Systems, Data Serialization Strategies

Target Audience & Prerequisites

The talk targets slightly experienced folks (1–2+ years of experience in backend) or even those who are early in their career if they have spent some time in systems thinking.

In terms of prerequisites, AWS has built a set of pretty good resources around caching: https://aws.amazon.com/caching/. The Systems Design Roadmap also provides a path to start more research around caching: https://roadmap.sh/system-design.

For those interested in a more advanced perspective, this paper is a very good entry point around patterns and behaviors observed when managing large-scale caching systems (both in terms of infrastructure and various use cases): https://www.usenix.org/conference/osdi20/presentation/yang

Slide Deck

Draft: https://docs.google.com/presentation/d/1L9oHgnhioVKZikgdf-v5918PXqWhk9uCF2Pg9QejVac/edit?usp=sharing

Draft V1: https://docs.google.com/presentation/d/155ppuYIijtpLSCJDA9jk7c1RFTjY1bFq28Z0sqoBpeo/edit?usp=sharing

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hybrid Access Ticket

Hosted by

We care about site reliability, cloud costs, security and data privacy

Supported by

Platinum Sponsor

Nutanix is a global leader in cloud software, offering organizations a single platform for running apps and data across clouds.

Platinum Sponsor

PhonePe was founded in December 2015 and has emerged as India’s largest payments app, enabling digital inclusion for consumers and merchants alike.

Silver Sponsor

The next-gen analytics engine for heavy workloads.

Venue host - Rootconf workshops

Community Partner

FOSS United is a non-profit foundation that aims at promoting and strengthening the Free and Open Source Software (FOSS) ecosystem in India. more

Community Partner

A community of rust-lang contributors and end-users from Bangalore. We also have presence on the following telegram channels https://t.me/RustIndia https://t.me/keralars https://t.me/fpncr Twitter handle: https://twitter.com/rustlangin more