Rootconf Delhi edition

On network engineering, infrastructure automation and DevOps

Tickets

PubSub Realtime messaging service @ Hotstar

Submitted by Piyush Gupta (@piyushgupta27) on Tuesday, 26 November 2019

Section: Full talk (40 mins) Category: Distributed systems

View proposal in schedule

Abstract

This talk covers our journey of building an MQTT based Pubsub system for 50M concurrent socket connections, the challenges faced and the architecture that powered Hotstar’s realtime social features for IPL 2019.

Outline

The Social & Gaming Team at Hotstar built an interactive Social Feed in VIVO IPL 2019 that appears below the video on the Hotstar mobile apps.

The content in the feed comes from various source, local timer objects, Questions/Answer/Prizes/Rounds/Advertisements/Celeb handles, API calls, user initiated and for a matter of fact, anything that can be shown on the feed in real-time without any scope of caching and without draining clients’ data/battery.

PubSub is a highly scalable and durable messaging infrastructure that serves as a foundation for realtime communication with millions of concurrent users. By providing one-to-many (broadcast or fan-out) use-cases as a starting point, PubSub delivers low-latency, durable messaging from various backend services to all connected users simultaneously with minimal battery and data usage.

Piyush Gupta will talk about his journey of building PubSub Infrastructure. He will stress upon the challenges faced and learnings accrued on this journey of building a system capable of handling 50M peak concurrent connections with 1rps messages sent rate. Over the duration of VIVO IPL 2019, this service ended up sending over 250 Billion+ messages.

Speaker bio

Full stack developer, Building Social for Hotstar

Links

Slides

https://docs.google.com/presentation/d/1-4-FOuvwEi7kAz7ZlD3RaF4syF2en49GGwFqFlnTDm8/edit#slide=id.g5bf29b27ba_1_64

Comments

  • Anwesha Sarkar (@anweshaalt) Reviewer 16 days ago

    Hello Piyush,

    Thank you for the submission. Here are the feedback for your talk :

    • Add a slide introducing you, it is important for the audience to know their speaker.
    • Add take away slide.
    • Add conclusion slide.
    • Instead starting with a agenda slide, can you start with the Origin story?
    • The slides look real text heavy, avoid that.
    • Divide the cluster test slide into 3 or 4 parts. Also do not name them as Cluster Test Continued.

    Submit the revised slides by 30th November, 2019 (latest). Look forward to your reply.

    Regards,
    Anwesha

  • Anwesha Sarkar (@anweshaalt) Reviewer 16 days ago

    Hello Piyush,

    Thank you for the submission. Here are the feedback for your talk :

    • Add a slide introducing you, it is important for the audience to know their speaker.
    • Add take away slide.
    • Add conclusion slide.
    • Instead starting with a agenda slide, can you start with the Origin story?
    • The slides look real text heavy, avoid that.
    • Divide the cluster test slide into 3 or 4 parts. Also do not name them as Cluster Test Continued.
    • “Piyush Gupta will talk about his journey of building PubSub Infrastructure.” - what do you mean by this - a. Is this a personal journey? or a hotstar journey which you were a part of? - which part you are stressing on.
    • If this is your personal journey then how will it help the audince?

    Submit the revised slides by 30th November, 2019 (latest). Look forward to your reply.

    Regards,
    Anwesha

  • Anwesha Sarkar (@anweshaalt) Reviewer 2 days ago

    Hi Piyush,

    Here are the feedback from yesterday’s rehearsal :

    1. The slides are too text heavy.
    2. Many slides including ( introduction slide and the next slide) look like a small blog post and we do not need this.
    3. The replaces the words “lingos and slangs” with the word terminology.
    4. Include more graphs with highlighting the key points on the graph.
    5. Set context before the reverse bridge slide/mention
    6. Do not start your talk with a ‘so’.
    7. Be gender neutral no usage of “Guys” .
    8. Include a take away slide.
    9. Include a conclusion slide.
    10. Add your contact credential from the end slide.
    11. Represent the problem statement through the pictorial version.
    12. Cut down the problem statement in simple words
    13. Spend some more time to explaining the problem.
    14. Slow down the pace.
    15. Include a link slide with blog link and other important links.

    Submit your revised slides by 18th December 2019.

    Regards,
    Anwesha

Login with Twitter or Google to leave a comment