Rootconf 2014

My aim is to describe the current best practices and software to use when creating an auto scaling infrastructure. I showcase an elastic infrastructure that is based on proven methods and open source software (Flipkart Hostdb and Puppet) which enables us to build a platform that inturn allows application engineers to create massively scalable web apps without losing sleep.

Outline

The state of auto scaling infrastructure is still in its infancy and information, open source tools and best practices to automatically scale infra and applications on a the cloud is scarce.

Flipkart.com has a private cloud spread over two datacenters, 5500 Servers, four environments. 350+ developers. Project based environments are created and destroyed automatically on demand, utilizing Infrastructure services viz Mail, DNS, Monitoring, Load Balancing, Backup and application services like Mysql, RabbitMQ, Couchbase. Software is developed and deployed using continuous integration.

How do we configure DNS automatically? How do we create Mysql clusters with master and slaves on demand? How do we make sure that all virtual machines come with monitoring and alerting by default? How are log files purged and backed up on their own? How do we make sure that your weekend hack is ready to perform at web scale; when the whole world discovers it at the same time? All these are important questions where automatic scaling, elastic infrastructure is concerned. I will answer all of the above.

My aim is to describe the current best practices and software to use when creating an auto scaling infrastructure. I showcase an elastic infrastructure that is based on proven methods and open source software (Flipkart Hostdb and Puppet) which enables us to build a platform that inturn allows application engineers to create massively scalable web apps without losing sleep.

The lessons we’ve learned demonstrate that a combination of state management and configuration management systems is effective in building and maintaining an infrastructure and applications that are elastically scalable. The ideas discussed here can be applied to a private cloud or a public cloud like AWS.

Requirements

Basic understanding of Virtualization and Puppet or Chef.

Speaker bio

I’m the Master of Disaster at Flipkart, where I work on creating and maintaining services that operate at web scale, Flipkart is India’s largest e-commerce website with multiple data centers and over thousands of servers. Before coming to Flipkart, I was a senior performance engineer at Yahoo! I’ve spoken at numerous conferences and events.

Slides

https://speakerdeck.com/spo0nman/building-elastic-infrastructures