Rootconf Mini 2024

We shall discuss Drove, a simple container orchestrator developed at PhonePe that focuses on efficient resource utilization, container performance, straightforward compliance and security models, and ease of management. At PhonePe, containers running on Drove clusters, deployed across our multiple Data Centers and cloud, handle millions of requests per second and power all services and apps across the different business lines.

Drove is focussed towards reliable and easy management of stateless and containerised services and tasks at large scale.

We shall cover how Drove helps us manage isolated clusters running interdependent microservices across compliance firewalls, the Drove Gateway, a traffic gateway utilizing Nginx auto-configuration, the Drove coredns plugin for service discovery, as well as Epoch, our simple time-based job scheduler built on Drove.

Agenda

• Why we developed Drove
• Applications and tasks in Drove
• The Drove cluster
• Drove Gateway
• Drove CoreDNS plugin
• Epoch - our time based job scheduler built on Drove
• Operations and maintenance (including caveats on docker and podman)
• Observability of the drove cluster and containers
• Introduction to the drove repository and documentation

Takeaways

• Drove provides a simplified orchestration system to deploy and execute service containers across a cluster of machines. It is a logical successor to Apache Mesos/Marathon based container orchestration system.
• It does not strive to solve all requirements for building the complete infrastructure layer in a DC/Cloud. Instead, some other requirements, such as service discovery, auto-scaling, and key-value storage, are handled by existing platforms at PhonePe.. It is decidedly simpler than systems like kubernetes and openshift, provides a simple deployment architecture with very few moving components while steering clear of adding too much bloat to the system.
• Battle tested on production with clusters running thousands of containers that handle millions of requests per second across hundreds of hosts per cluster spanning multiple data centers
• Open sourced end to end

Audience

• Site Reliability and DevOps Engineers
• Engineering leaders
• Cloud architects and engineers

For further resources visit https://phonepe.github.io/drove-orchestrator/