Speak at The Fifth Elephant 2026 Annual Conference

Cross-business-unit data sharing usually starts with good intentions and quickly turns into ticket-driven exports, undocumented copies, governance bottlenecks, and growing compliance risk. At InMobi, multiple business units operate independent lakehouses, catalogs, and data engineering organizations. Combining data across these domains creates significant business value, but traditional approaches either require centralized warehouses that undermine domain ownership or ad-hoc data movement that creates governance, operational, and compliance challenges.

To address this, we built DataBridge, a governed cross-business-unit data sharing platform based on a simple principle: Discover Globally, Materialize Locally.

DataBridge enables governed exploration through a unified metadata catalog and federated query layer built on Trino, Polaris, and OpenMetadata. Teams can discover and analyze datasets in place using short-lived credentials, policy-enforced access controls, and bounded exploratory queries—without moving data or giving up ownership of their domains. Only when a use case demonstrates repeatable value does the consumer request materialization into their own environment, where governance, legal, compliance, and lifecycle management processes are applied.

In this talk, we’ll share the architectural principles behind the platform, the decision framework used to choose between federation and materialization, and the lessons learned from operating governed cross-domain data products at scale across both Delta and Iceberg ecosystems.

We’ll dive into the metadata-consistency challenges that emerge when data movement is replaced by metadata movement, including catalog synchronization, UniForm adoption, schema evolution, and operating a federated query layer across Delta and Iceberg ecosystems. We’ll also discuss how incremental materialization pipelines identify and propagate only changed partitions instead of repeatedly copying entire datasets, enabling scalable cross-domain data products while preserving ownership boundaries.

Finally, we’ll share the production bottlenecks that emerged while operating the platform at scale and the patterns that kept it reliable without sacrificing governance or domain ownership.

Key Takeaways

1. Separating Discovery from Data Movement

Learn how governed, in-place discovery enables teams to validate business value before creating long-lived data copies. We’ll share the practical signals we use—query frequency, latency requirements, cost-per-scan, freshness expectations, and compliance considerations—to decide when a consumer should continue federating versus materialize data into their own domain.

2. Operating Delta and Iceberg Under a Shared Data Plane

Understand the operational realities of running Delta and Iceberg side by side through a shared catalog and federated query layer. We’ll cover metadata synchronization strategies, UniForm adoption lessons, schema evolution challenges, cross-domain access control, and the production issues that emerge when multiple lakehouse technologies must behave like a single platform.

3. Building Incremental Materialization at Scale

See how metadata-driven change detection, partition-level incremental processing, and consumer-driven materialization enable scalable cross-domain data products without creating uncontrolled data sprawl. We’ll discuss the tradeoffs, failure modes, and operational lessons from running these workflows in production.

Audience

This session is aimed at Data Engineers, Data Platform Engineers, Data Architects, and Engineering Leaders building Data Mesh, Data Product, or federated analytics platforms. It will be particularly valuable for teams dealing with cross-domain data sharing, governance, metadata management, and lakehouse architectures built on technologies such as Iceberg, Trino, Polaris, OpenMetadata, and Delta Lake.

Bio

Shivam Gupta is a Staff Engineer at InMobi, architect of DSP, InMobi’s real-time bidding platform, & lead of the materialization platform of the DataBridge data mesh.

Deep Patel is a Staff Engineer 2 at InMobi, lead architect of the DataBridge data mesh & 1Weather, InMobi’s Climate Intelligence Platform.

Kuldip Puri Tejaswi is a Senior Data Engineer on InMobi’s central reporting platform team, leading the query federation layer of the DataBridge data mesh.

Slides Link -> https://docs.google.com/presentation/d/1OiFhrpu87Gq6eJkJn1tNVdWLhqfmlZi1H4QwDwCXV-M/edit?usp=sharing