Snehasish Roy

@snehasishroy

Clockwork: The Backbone of PhonePe’s 2 Billion Daily Jobs

Submitted Oct 28, 2024

Overview

Have you ever had an alarm fail to wake you up, causing a ripple effect of chaos in your morning? At PhonePe, we understand the criticality of such ‘alarms’ in our digital ecosystem.

Take, for instance, our daily Merchant Settlements process. A merchant receives multiple transactions during the day. At the end of the day, we want to ensure the final amount gets credited to their account. A potential delay in this routine job being executed means a merchant not receiving their earnings on time, shaking their trust as a PhonePe Customer.

At PhonePe, we face the colossal task of managing over 2 billion daily callbacks. The ability to handle over 100,000 job schedules per second with single-digit millisecond latency is not just a goal; it’s a necessity. At p99, our system ensures that there’s no lag in job execution, which in the worst case is capped at 1 minute. Follow along to learn how we’ve made this possible.

In this talk, we will take a look at the internals of Clockwork – the system that powers job scheduling across various teams at PhonePe and enables clients to easily onboard and schedule future jobs without the need for heavy lifting on their own.

Agenda

  • Why we developed Clockwork
  • Deep dive into design and architecture of providing Callbacks at scale (100k callbacks per second)
  • Tech stack : Hbase, RMQ, Zookeeper

Takeaways

  • How Zookeeper helps in achieving consensus in distributed systems.
  • Do’s/Dont’s for managing infrastructure at scale.

Audience

  • Site Reliability and DevOps Engineers
  • Engineering leaders
  • Cloud architects and engineers

For further resources visit https://tech.phonepe.com/clockwork-the-backbone-of-phonepes-2-billion-daily-jobs/

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hosted by

We care about site reliability, cloud costs, security and data privacy

Supported by

Platinum Sponsor

Nutanix is a global leader in cloud software, offering organizations a single platform for running apps and data across clouds.

Platinum Sponsor

PhonePe was founded in December 2015 and has emerged as India’s largest payments app, enabling digital inclusion for consumers and merchants alike.

Silver Sponsor

The next-gen analytics engine for heavy workloads.

Sponsor

Community sponsor

Peak XV Partners (formerly Sequoia Capital India & SEA) is a leading venture capital firm investing across India, Southeast Asia and beyond.

Venue host - Rootconf workshops

Thoughtworks is a pioneering global technology consultancy, leading the charge in custom software development and technology innovation.

Community Partner

FOSS United is a non-profit foundation that aims at promoting and strengthening the Free and Open Source Software (FOSS) ecosystem in India. more

Community Partner

A community of Rust language contributors and end-users from Bangalore. We have presence on the following telegram channels https://t.me/RustIndia https://t.me/fpncr LinkedIn: https://www.linkedin.com/company/rust-india/ Twitter (not updated frequently): https://twitter.com/rustlangin more