The Fifth Elephant 2026 Annual Conference

Description

Query engines are amongst the most interesting pieces of software, they span from frontend to the lowest layers of software inside hardware! They have their own compiler, graph theory applications, truly distributed systems, low level kernel and even hardware, you name it and there’s a variant present. But this has also meant, teams working on these behemoths have to be really good at understanding every part of the stack. This means a new team wanting to disrupt the market needs to build everything from scratch, covering everything functionally and after all that, nail the actual selling point of their product. But there has been a shift in outlook on query engines in recent times and database folks have started making different parts of query engines composable. Starting with strong fundamentals of vector memory layouts, storage representations, SIMD optimized functions, extensible operators, extensible query optimisers, every part is getting built and polished to precision by a purpose specific project. Datafusion, Calcite, Arrow, Parquet, etc are some of the leading examples in this space. In this talk, we will explore some of these abstractions and the beauty which has allowed them to dominate the ecosystem. Whether the future of databases is monolithic systems or specialised databases, composability is here to stay.

Takeaways

• Understanding examples of composability by looking at APIs from Arrow, datafusion, parquet etc
• How a truly composable query engine pushes language and even kernel boundaries?
• Industrial adopters leveraging these composable systems to show their robustness
• How can one leverage these ecosystems incrementally in current systems?

Who should attend?

If you are a person interested in/actively working in query engine space or systems work in general, this talk is for you!

Bio

Fenil Jain is part of query engine team at e6data working on their OLAP engine. E6data’s engine is a distributed database designed to handle queries churning data at monster scale!