Speak at Bengaluru Systems meet-up

Every Cassandra cluster carries a body of shared state that every node must agree on to function correctly — this is cluster metadata. It includes the distributed schema (keyspaces, tables, types, and functions), the cluster topology and membership, and, most critically, token ownership: the mapping that decides which nodes are the read and write replicas for any given range of data. Unlike the user data Cassandra stores, this metadata is the shared source of truth that tells every coordinator where data lives and how to reach a correct quorum. When nodes disagree about it, they don’t just serve stale reads — they can silently violate the consistency and durability guarantees the database promises.

Historically, Cassandra propagated these changes through gossip, which favors eventual consistency and resilience. Gossip is probabilistic, unordered, and slow to converge, and this led to a number of issues for operators and users. On the topology side, because there is no global order for changes, two coordinators can end up with different views of the ring during a bootstrap, decommission, or move. This can produce disagreeing read/write replica sets and, in some cases, transient or even permanent data loss. To work around this, Cassandra relied on timing-based throttling (RING_DELAY) and on avoiding concurrent range movements, which made topology operations slower and harder to reason about. Schema changes had a separate but related weakness: unordered, last-write-wins schema mutations allowed racing operations to leave a cluster in an inconsistent state — for example, a CREATE TABLE racing a DROP TYPE, or concurrent CREATE TABLEs producing conflicting table IDs. These aren’t rare edge cases; they touch some of the most fundamental operations of running a cluster.

Transactional Cluster Metadata (CEP-21) reframes cluster metadata as a problem of linearizable, ordered state rather than probabilistic dissemination. At its heart is the Cluster Metadata Service, which maintains a single, immutable, totally-ordered log of metadata change events, each stamped with a monotonically increasing epoch. Every change is validated against current state, committed via consensus, and then made visible in a strongly consistent way — so complex operations like bootstrap or decommission become deterministic, step-wise sequences that safely stage pending states and can be resumed or cancelled. Crucially, nodes exchange their current epoch on ordinary messages, so a lagging coordinator is detected and caught up at read/write time. This guarantees that no stale node can ever assemble a quorum inconsistent with the true ownership of a range. Philosophically, TCM trades away the assumption that the cluster must tolerate perpetual disagreement, and instead gives Cassandra a single reference version of the truth — replacing gossip’s “eventually, maybe” with “in this exact order, verifiably” — and creates the foundation for future work like load-aware, dynamic data placement.