Rootconf
Rootconf

Rootconf topical edition on Databases

Call for Proposals

All submissions
Vivek Sinha

Vivek Sinha

@vivekwiki

High Concurrency & Low Latency Serving on Apache Iceberg

Submitted May 25, 2026

Session type - select the format for your session: 30-minute talk – technical deep dive

Description

Everyone is putting their data into Apache Iceberg; almost no one is serving sub-second queries directly from it. Once data lands in Iceberg, a familiar question arises: how do you power real-time experiences without duplicating it in yet another serving system? This challenge is especially sharp in observability workloads like RUM, clickstream, and APM, and in customer-facing analytics dashboards, where the cost of low latency often leads to pipeline sprawl. Data teams want strong isolation and fast queries, but not a web of ETL jobs and redundant data copies.

In this talk, I’ll share how we at StarTree built a fast query layer for Apache Iceberg using Apache Pinot, and what it takes to make Iceberg behave like a low-latency analytics store at scale. I’ll walk through a Kafka to Iceberg to Pinot/StarTree architecture where Iceberg stays the single source of truth while Pinot powers production queries. The session covers the key design decisions: Pinot indexing and pruning for selective reads, parallel prefetching of Iceberg blocks over S3, and trading off local vs remote storage for cost and latency. I’ll close with benchmark results on roughly 1 TB across realistic query shapes and real-world examples, both internal customer metrics exploration and external product analytics, all without extra ETL or duplicate data.

Takeaways

  1. A practical blueprint for adding a low-latency serving layer on top of Apache Iceberg without data duplication or additional ETL pipelines.
  2. A reference Kafka to Iceberg to Pinot/StarTree architecture with concrete design choices and benchmark data (QPS and latency on roughly 1 TB) for filters and aggregates across primitive and complex types.

Who Should Attend

Data engineers, platform engineers, and backend engineers building real-time or user-facing analytics products who are evaluating or already using Apache Iceberg as a data lakehouse foundation.

Bio

Vivek Sinha leads the initiative to extend Apache Pinot’s query engine to Apache Iceberg at StarTree, making high-concurrency, low-latency serving possible directly on open lakehouse data.

He brings over a decade across database systems, ETL, data lakes, and OLAP, having worked as a founding engineer and product leader at companies like Hevo Data on both batch and real-time processing at Fortune 500 scale. He’s a frequent speaker at data infrastructure events, where he digs into the architecture and tradeoffs behind real-time analytics.

You can find him here at LinkedIn, Twitter, GitHub

All submissions

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hosted by

Rootconf
Rootconf
We care about site reliability, cloud costs, security and data privacy