The Fifth Elephant round the year submissions for 2019

Submit a talk on data, data science, analytics, business intelligence, data engineering and ML engineering

Propose a session

A Journey of Building Dream11's Data Platform

Submitted by Pradip Thoke (@thokepradip) on Friday, 10 May 2019

Abstract

Dream11 is India’s biggest fantasy sports platform that allows users to play fantasy cricket, hockey, football, kabaddi and basketball.
Our total user base is over 70 million and expected to cross 100 million by end of 2019.

2019 IPL, Dream11 served 2.4M concurrent users, 2200/sec contest joins, 5k/sec team save/edits.
On a data size front, we are generating 2TB per day transactional and events data volume.

We started small in 2008 and grew exponentially every year, we would like to share our journey of building day1 startup data platform to a more scalable data platform aligned with our growth pattern supporting real time operational and batch analytics use cases.

Outline

  • Day 1 Startup Data Pipeline for Analytics
  • Data Collection Framework
  • Data Lake Architecture
  • Integrated Data Warehouse
  • Real Time Analytics and Alerting
  • Data Quality
  • Learnings

Speaker bio

Pradip has been in Data Engineering space for 12+ years and currently working as an Architect - Data Engineering at Dream11. Throughout the years, he has worked extensively on bulding real time, batch data analytics pipelines and Data Warehouse implementations in traditional and modern big data platforms on both on-premises and on the cloud. He has a Masters from BIT’s Pilani, Rajasthan.

Comments

  • Abhishek Balaji (@booleanbalaji) Reviewer 5 months ago

    Hi Pradip,

    This seems to be a duplicate of the proposal you had submitted earlier. (https://hasgeek.com/fifthelephant/2019/proposals/a-journey-of-building-dream11s-data-platform-DXFDqHSFe4FZqcFRX3ky7Z). We cannot evaluate the proposal without the preview video and slides.

    Please add the updated slides and preview video. Your slides must cover the following:

    • Problem statement/context, which the audience can relate to and understand. The problem statement has to be a problem (based on this context) that can be generalized for all.
    • What were the tools/options available in the market to solve this problem? How did you evaluate these, and what metrics did you use for the evaluation? Why did you decide to build your own solution?
    • Why did you pick the option that you did?
    • Explain how the situation was before the solution you picked/built and how it changed after implementing the solution you picked and built? Show before-after scenario comparisons & metrics.
    • What compromises/trade-offs did you have to make in this process?
    • What are the privacy, regulatory and ethical considerations when building this solution?
    • What is the one takeaway that you want participants to go back with at the end of this talk? What is it that participants should learn/be cautious about when solving similar problems?

    We need to see your updated slides and video on or before 21 May. If these are not updated, we’d have to consider the proposal for a future event.

  • Abhishek Balaji (@booleanbalaji) Reviewer 4 months ago

    Marked as rejected since proposer hasnt responded to comments/updated content before deadline. Will be considered for a future event if content is updated.

Login with Twitter or Google to leave a comment