From Nginx Bottlenecks to Nexus: Our In-House Service Mesh for Scale and Savings

Submitted Apr 11, 2025

Topic of your submission: Kubernetes Type of submission: 30 mins talk I am submitting for: Rootconf Annual Conference 2025

Description

As our microservices architecture expanded, manual Nginx configurations and lack of service discovery created bottlenecks. We built our own service mesh using Envoy as a proxy, and the impact it’s had on our company has been incredible—improving observability, lowering the tail latency, saving 95% NIZE costs, improving success rates, at the scale of 7.5 million requests/sec.

In this session, we’ll share our experience of building a service mesh from scratch, diving into the architectural decisions we made, the roadblocks we hit, and the optimizations we applied. We’ll also talk about why we chose to build our own solution instead of going with an off-the-shelf service mesh. Our goal is to share the lessons we learned—both the wins and the challenges—so the community can benefit from our journey and avoid some of the mistakes we made along the way.

Benefits to The Ecosystem

This session will provide deep insights into:

How we built a service mesh from scratch—the key architectural decisions, challenges, and optimizations.
Cost-saving strategies that reduced infrastructure expenses through better load balancing and network routing.
Scaling best practices for managing millions of requests per second with minimal overhead.
The build vs. buy decision process—when it makes sense to create your own solution vs. adopting an existing one.

This talk provides a first-hand account of building a service mesh from scratch — real-world lessons, engineering insights, and concrete strategies for scaling efficiently. Whether you’re considering implementing a service mesh or looking to optimize an existing one, this session will equip you with actionable takeaways to improve your system’s performance, resilience, and cost efficiency.

Rootconf 2025 Annual Conference CfP