The Fifth Elephant 2019

Dream11 is India’s biggest fantasy sports platform that allows users to play fantasy cricket, hockey, football, kabaddi and basketball.
Our total user base is over 50 million and expected to cross 100 million by end of 2019.

2018 IPL, Dream11 served 800k concurrent users, 1023/sec contest joins, 1400/sec team creations.
On a data size front, we are generating 700GB per day transactional and events data volume.

We started small in 2008 and grew exponentially every year, we would like to share our journey of building day1 startup data
platform to a more scalable data platform aligned with our growth pattern supporting real time operational and batch analytics use cases.

Outline

• Day 1 startup data pipeline Data Warehouse 1.0
• Data collection framework
• Data Lake Architecture
• Integrated Data Warehouse 2.0
• Real Time Operational and batch analytics
• Data Quality
• Learnings

Speaker bio

Pradip has been in Data Engineering space for 12+ years and currently working as an Architect - Data Engineering at Dream11. Throughout the years, he has worked extensively on bulding real time, batch data analytics pipelines and Data Warehouse implementations in traditional and modern big data platforms on both on-premises and on the cloud. He has a Masters from BIT’s Pilani, Rajasthan.