Rootconf 2025 Annual Conference CfP

A customer running a Python Django app with Gunicorn on Kubernetes faced growing complaints about slow response times.

They used KEDA (Kubernetes Event Driven Autoscaler) to address this problem. But instead of spending time to explore the right trigger metric, they chose what was easy to measure - CPU load.

The problem? Not all requests were CPU-bound. The bottlenecks were spread across memory, network, database calls, making CPU-based scaling ineffective. This led to frequent pod churn, overprovisioning, and rising infra costs.

The customer checked their AWS bills and had the following queries:

• By what factor is the current solution overprovisioning the scale out?
• What is the right metric to scale out on to cut these costs?

While trying to find the right metric, we ran into these challenges:

1. The Gunicorn PR with the ideal scale out metric had been stuck in review for the last few years.
2. Their current monitoring system (Datadog) had limitations which prevented timely scale out.

I will share some interesting lessons and best practices we realized while solving this problem:

• Identifying workload bottlenecks before scaling.
• Navigating trade-offs with interim, suboptimal solutions.
• Discovering the right math to quantify overprovisioning.
• Deciding the scaling criteria within the constraints imposed for efficient and cost-effective autoscaling.

I will also demo the final solution, which used fundamental Linux monitoring techniques to identify the optimal scaling trigger.

This talk will be beneficial to developers, DevOps engineers and SREs.

Bio:

I am Saurabh Hirani, a Principal SRE at One2N with over 18 years of experience in backend engineering and infrastructure automation.

I have worked with startups and enterprises, helping them design scalable, reliable systems. I enjoy sharing my learnings through talks and writing.