PWL Oct 2023: "Keep CALM and CRDT On"

About the paper

Can we achieve distributed exeuction in storage systems without the need for coordination? We have a myriad of coordination protocols ranging from 2PC in transaction processing, to consensus protocols like Paxos and Raft. However, the scalability of a system is limited by how much coordination it needs to perform. Not just its scalability, but also its availability is impacted by this. However, is coordination a nescessary condition for correctness? We have the CALM theorem result to reason about distributed executions in a coordinated free manner, and more importantly, what classes of programs can be executed without coordination.

CRDTs are one such implementation that utilise these properties to provide an API that does not require coordination, however, there are subtle ‘gotchas’ to these data structures. This paper describes these footguns and formalises a query model over CRDTs so that they can be used with confidence and reliability.

The paper is published here: https://www.vldb.org/pvldb/vol16/p856-power.pdf

Key takeaways for the audience

• Gain insight into classes of distributed computation that can be performed without the need for coordination
• Learn about a formal method of thinking about coordination-free execution
• Learn about foot-guns involved with working with Conflict Free Replciated Datatypes (CRDTs) and how those can be taken care of using a query model for CRDTs.

About the presenter

Madhav Jivrajani is a maintainer of the Kubernetes project where he deals with areas of Architecture, API Machinery and Scalability. His interests include formalising distributed systems and improving their performance at the micro and macro scale, analytical modelling methods like queueing theory, and tangential intersections of fields with distributed systems, most recently intersection of philopsophy, open source communities and systems.

He is also a Technical Lead for the Contributor Experience Special Interest Group of Kubernetes, and a GitHub Admin for the project.

More about him: https://nonmonotonic.dev/

RSVP and venue

This is an in-person meetup. RSVP to get the venue location.