Trends in Data Engineering

👩🏻 💻 About the Meetup 👩🏻 💻

This in-person meetup was held on 31 January, 2025 at the Koramangala Club.

50 participants registered for the meetup; 30 participants were present at the meetup, including engineers with 4+ years experience, and architects.

The Fifth Elephant meetup with Achal Shah

Achal Shah discussed the evolution of compute engines and data architectures. He covered the shift from distributed processing engines like Spark to single node OLAP engines like DuckDB, and the move towards lakehouse architectures that combine the best of data lakes and data warehouses. Achal also touched on the rise of Python as the de facto API for data ecosystems, and the trend towards building focused end-user applications rather than platforms. He concluded by speaking about real-time machine learning and MLOps, and the challenges of data cleaning and feature engineering

Sandeep Joshi - founder at Kognitos, and editor of data engineering track at The Fifth Elephant - curated the meetup, and moderated the discussion.

Sandeep and Achal at the meetup

Achal has worked at the intersection of ML Infrastructure and Systems for years now, focused on real-time and streaming systems. After the discussion, participants networked with each other.

Participants interacting after the presentationNetworking at the meetup

About Achal Shah

Achal is Tech Lead Manager (TLM) at Tecton.

🔰 Sponsorship Credits 🔰

This meetup was open to anyone who wants to attend. Numberz.ai sponsored the meetup by hosting it, and arranging for Achal’s stay in Bangalore.

Contact information

For inquiries,
☎️ Call - (91)7676332020
📧 Email - info@hasgeek.com
💬 Leave a comment at https://hasgeek.com/fifthelephant/fifthel-january-2025-meetup/comments

Venue

Koramangala Club

Room no. #203, 6th Cross Rd,

6th Block, Koramangala,

Bengaluru, - 560095

Karnataka, IN

Loading…

Hosted by

Jump starting better data engineering and AI futures

Supported by

Meetup Sponsor

Numberz.ai is a collaboration and knowledge management platform that connects teams, streamlines communication, and enhances decision-making through advanced AI