Sujeet Gholap

Architecting Observability Platform on S3 + Lambda

Submitted Jun 25, 2026

Observability is a Data Engineering problem. Write-heavy, realtime latency, faster queries are the typical requirements of a Observability system. Traditional observability systems fail to meet the ever-growing demand of increased volumes due to cloud deployments, AI agents speeding up the feature and product development. AI agents also changes the query patterns which were common in Observability - human-led sparse queries. AI agents can make 100x more queries than human engineers which breaks the assumptions underlying fixed compute and disk based observability architectures. This session goes into building blocks required to design an observability system on Object storage and Serverless architecture.

Key Takeaways

  1. An architectural blueprint of an observability system on top of object storage and serverless compute. It includes how to design data file formats and index file formats optimized for S3 dynamics.
  2. Utilizing serverless compute for bursty loads is essential for reducing the idle tax of always-on clusters.

Target Audience

This session is ideal for

  • Data engineering architects who are building scalable and performant systems on object storage.
  • SRE/DevOps engineers navigating cost / operational overhead challenges of traditional observability systems

Bio

Founding Engineer at Oodle AI, an observability platform built on object storage + serverless compute.

Prev: Early (among fist 30 hires) engineer at Rubrik

sujeet.gholap@oodle.ai

Draft slides

to be added

Elevator pitch:

to be added

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hosted by

Jumpstart better data engineering and AI futures