Session Description

Modern LSM-tree storage engines typically force a global choice between tiered and leveled compaction. Tiered compaction offers excellent write throughput but can suffer from read amplification, while leveled compaction improves read performance at the cost of additional write amplification. Existing systems generally apply one strategy across the entire database, implicitly assuming that all data exhibits similar access patterns and workload characteristics.

Amethyst explores a different approach: treating compaction as a local rather than global decision. The system continuously characterizes SSTable behavior using lightweight metadata and dynamically selects between compaction strategies at the segment level. In this talk, we will examine the trade-offs that motivated the design, the challenges of workload characterization, the mechanisms required to safely transition between policies, and the results from benchmarking adaptive compaction against traditional LSM configurations. We will also discuss cases where adaptation helps, where it fails, and what these results suggest about the future of self-tuning storage engines.

Key Takeaways

Understand why the traditional choice between tiered and leveled compaction remains one of the fundamental trade-offs in LSM-tree design.
Learn how lightweight workload characterization can enable adaptive compaction policies and the engineering challenges involved in building self-tuning storage systems.

Target Audience

This session will be valuable for:
Database engineers and storage engine developers
Distributed systems practitioners
Performance and infrastructure engineers
Researchers and students interested in storage systems and database internals
Anyone operating or evaluating LSM-based systems such as RocksDB, Cassandra, ScyllaDB, LevelDB, or CockroachDB

Speaker Bio

Suchitra is a Computer Science student with interests in databases, distributed systems, and storage engines. She’s currently building Amethyst, an experimental LSM-tree storage engine that investigates adaptive compaction strategies and workload-aware storage optimization. Her work focuses on bridging ideas from database research and practical systems engineering through hands-on implementation and benchmarking.

Nilin is a Software Engineering and Computer Science student specializing in backend systems and database internals.As a co-developer of Amethyst, she specializes in implementing adaptive compaction logic, physical disk I/O, and performance benchmarking.