Rootconf
Rootconf

Rows, columns, and consequences

Speak at Rootconf’s Special Edition on Databases

All submissions

Nitesh Vijay

Multi-Tenant Vector Search: The Indexing Nightmare Nobody Warned You About

Submitted Mar 15, 2026

Session type - select the format for your session: 30-minute talk – technical deep dive

You have 10,000 tenants sharing one database. Each tenant wants vector search over their own data. Do you build one giant index and filter at query time? Or 10,000 small indexes that each need their own memory budget? Both answers are wrong in interesting ways.

This talk is about the multi-tenancy problem in vector search, which is quietly one of the hardest unsolved problems in the space. I work on Azure Cosmos DB at Microsoft, where multi-tenant workloads are the norm. Every architectural choice we made for vector indexing had to survive the question “okay, but what happens when 10,000 customers do this simultaneously?”

I’ll walk through three approaches we tried and the tradeoffs of each. Global index with tenant filtering: fast to build, terrible isolation, and one noisy tenant can ruin recall for everyone. Per-tenant partitioned indexes: great isolation, but the memory overhead per index is brutal when you have thousands of tenants with tiny datasets. And the hybrid approach we landed on: partition-scoped DiskANN indexes that scale with the tenant’s data size without pre-allocating resources for tenants who barely use search.

Takeaways

  1. A framework for deciding between shared vs. isolated vector indexes based on your tenant count, data distribution, and latency requirements. With actual numbers from production workloads.
  2. Three specific failure modes of multi-tenant vector search that don’t show up in benchmarks but will absolutely show up in production: the noisy neighbor recall problem, the cold-start index problem, and the “one tenant deleted everything” index corruption problem.

Who is this for?

Database engineers and backend developers building SaaS products that need vector search. Platform engineers designing multi-tenant data layers. Anyone who’s tried to add semantic search to an existing multi-tenant application and hit a wall.

About me

Nitesh Vijay, Senior Software Engineer at Microsoft working on Azure Cosmos DB. I work on the vector search and indexing infrastructure that powers multi-tenant AI workloads at global scale. BITS Pilani alum.

All submissions

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hosted by

Rootconf
Rootconf
We care about site reliability, cloud costs, security and data privacy