PWL Oct 2023: "Keep CALM and CRDT On"
Keep CALM and CRDT On, ready by Madhav Jivrajani
About the paper
Can we achieve distributed exeuction in storage systems without the need for coordination? We have a myriad of coordination protocols ranging from 2PC in transaction processing, to consensus protocols like Paxos and Raft. However, the scalability of a system is limited by how much coordination it needs to perform. Not just its scalability, but also its availability is impacted by this. However, is coordination a nescessary condition for correctness? We have the CALM theorem result to reason about distributed executions in a coordinated free manner, and more importantly, what classes of programs can be executed without coordination.
CRDTs are one such implementation that utilise these properties to provide an API that does not require coordination, however, there are subtle ‘gotchas’ to these data structures. This paper describes these footguns and formalises a query model over CRDTs so that they can be used with confidence and reliability.
The paper is published here: https://www.vldb.org/pvldb/vol16/p856-power.pdf
Key takeaways for the audience
- Gain insight into classes of distributed computation that can be performed without the need for coordination
- Learn about a formal method of thinking about coordination-free execution
- Learn about foot-guns involved with working with Conflict Free Replciated Datatypes (CRDTs) and how those can be taken care of using a query model for CRDTs.
About the presenter
Madhav Jivrajani is a maintainer of the Kubernetes project where he deals with areas of Architecture, API Machinery and Scalability. His interests include formalising distributed systems and improving their performance at the micro and macro scale, analytical modelling methods like queueing theory, and tangential intersections of fields with distributed systems, most recently intersection of philopsophy, open source communities and systems.
He is also a Technical Lead for the Contributor Experience Special Interest Group of Kubernetes, and a GitHub Admin for the project.
More about him: https://nonmonotonic.dev/
RSVP and venue
This is an in-person meetup. RSVP to get the venue location.
Hosted by
What was the last paper within the realm of computing you read and loved? What did it inspire you to build or tinker with? Come share the ideas in an awesome academic/research paper with fellow engineers, programmers, and paper-readers. Lead a session and show off code that you wrote that implement… more
Supported by
Venue host
Obvious is an award-winning product and strategy consultancy that creates digital experiences across strategy, design, mobile applications and web technologies that our partners’ customers love to use. Over the last eight years, we’ve employed technology as a force-multiplier to build disruptive, b… more
About the paper
Can we achieve distributed exeuction in storage systems without the need for coordination? We have a myriad of coordination protocols ranging from 2PC in transaction processing, to consensus protocols like Paxos and Raft. However, the scalability of a system is limited by how much coordination it needs to perform. Not just its scalability, but also its availability is impacted by this. However, is coordination a nescessary condition for correctness? We have the CALM theorem result to reason about distributed executions in a coordinated free manner, and more importantly, what classes of programs can be executed without coordination.
CRDTs are one such implementation that utilise these properties to provide an API that does not require coordination, however, there are subtle ‘gotchas’ to these data structures. This paper describes these footguns and formalises a query model over CRDTs so that they can be used with confidence and reliability.
The paper is published here: https://www.vldb.org/pvldb/vol16/p856-power.pdf
Key takeaways for the audience
- Gain insight into classes of distributed computation that can be performed without the need for coordination
- Learn about a formal method of thinking about coordination-free execution
- Learn about foot-guns involved with working with Conflict Free Replciated Datatypes (CRDTs) and how those can be taken care of using a query model for CRDTs.
About the presenter
Madhav Jivrajani is a maintainer of the Kubernetes project where he deals with areas of Architecture, API Machinery and Scalability. His interests include formalising distributed systems and improving their performance at the micro and macro scale, analytical modelling methods like queueing theory, and tangential intersections of fields with distributed systems, most recently intersection of philopsophy, open source communities and systems.
He is also a Technical Lead for the Contributor Experience Special Interest Group of Kubernetes, and a GitHub Admin for the project.
More about him: https://nonmonotonic.dev/
RSVP and venue
This is an in-person meetup. RSVP to get the venue location.
Hosted by
What was the last paper within the realm of computing you read and loved? What did it inspire you to build or tinker with? Come share the ideas in an awesome academic/research paper with fellow engineers, programmers, and paper-readers. Lead a session and show off code that you wrote that implement… more
Supported by
Venue host
Obvious is an award-winning product and strategy consultancy that creates digital experiences across strategy, design, mobile applications and web technologies that our partners’ customers love to use. Over the last eight years, we’ve employed technology as a force-multiplier to build disruptive, b… more