Rootconf 2025 Annual Conference - 16th and 17th May

Analytics engine are hot these days and they keep getting better for various reasons. Table formats, object storage, network improvement, better hardware have all contributed to the ecosystem of analytical databases’ performance. Vector search on SQL engine is something we have built for e6data’s engine that takes all the primitives of analytical workloads optimised for the above and takes it to the world of semantic text search.

This talk will cover some basics of Vector search and have a brief showcase of how it works in e6data’ SQL engine. It will also see how an unified SQL and Vector search can power some scenarios that are often done in separate stages.

The talk will highlight aome key challenges that Vector search design poses, the concept of pre and post filtering, optimization techniques and how e6 attempts to solve for this.

Key Takeaways:

• Basics of design choices one must face when building a Vector Search engine, including optimizing CPU and Memory intensive operations.
• Storage formats and how they impact perf on Vector Search
• Vector indices - why are they so hot, and what are the tradeoffs especially in the lakehouse analytics world

Audience who’d be interested

• Data engineers/Lakehouse users
• Data architects
• Anyone curious about database internals

About me
Engineer at e6data, focussing on building out the Vector Search for e6 presently. Previously worked at Thoughtworks (Studios primarily)