Rust India Conference 2026

The Database Landscape:

Historically databases were monolithic and all the components of the database: the parser, optimizer, intermediate representation, file formats and the execution engine were made from scratch by the core team making the database. Examples of this model include PostgreSQL, SQLite and DuckDB. With the need for more specialized OLAP databases in specialized domains such as time series, observability, analytical, streaming, geospatial, the current trend is the breakout of OLAP components into standalone services. This trend is fueled by two major disruptions: One, Apache Arrow with its language-independent columnar memory format and execution primitives. Two, execution engine libraries like Meta’s Velox (C++), and Apache Datafusion (Rust).

Why is Rust a game changer for databases:

The world’s database engineers suddenly started speaking the same dialect of Rust. What has led to this? It’s not just the language - but also the ecosystem. The reasons range from the foundations of a database being standardized due to Cargo, Arrow-rs etc, fearless Cross-Company collaboration due to the safety features of rust and powerful native interop from other languages like Java. Rust language features like Enums (Algebraic Data Types) and Pattern Matching are very useful when writing query optimizers. Lifetimes and Ownership provide "Local Reasoning. We get “Deterministic Performance” with Rust - makes it easier to handle the dreaded tail latencies. The Rust ecosystem (specifically crates like std::simd or arrow-rs kernels) has made “auto-vectorization” and explicit SIMD instructions much more accessible to the “average” database engineer. Thanks to all these reasons, Rust has become the lingua franca for database engineering.

Who we are:

e6data is a lakehouse query engine primarily specializing in low latency high concurrency analytical queries on large data volumes, competing with Databricks and Snowflake on two fronts - performance and cost efficiency. Our engine was built from scratch in Java and was optimized for performance. For further performance improvements we started to use Apache Datafusion for our engine to utilize Arrow primitives and the OpenSource ecosystem around Datafusion while also contributing upstream.

What you will learn in this talk:

In this talk we will share our experience with Datafusion, the ease of use it offers and how well we leveraged the plug and play components along with our existing services. While the common assumption is that Rust would be faster than Java, our initial Rust engine was slower than the Java engine, we will explain the challenges we had to overcome to make our new engine faster than our Java engine. And finally we will talk briefly about the optimization we do in e6 apart from what comes in Datafusion out of the box to compete with Databricks and Snowflake.

Speaker Bio

Sudarsan Lakshmi Narasimhan is a founding engineer and the Head of Performance & ResearchEngineering at e6data. Nimalan is a Senior engineer in e6data working on the core engine.