Rootconf 2016
Rootconf is India's principal conference where systems and operations engineers share real world knowledge about building resilient and scalable systems.
UD
Umang Dhawan
@umangd
DevOps for Big Data infrastructure on the cloud
Submitted Feb 25, 2016
Section:
Crisp talk
Technical level:
Beginner
To discuss the challenges faced in managing and deploying to hadoop clusters on the cloud and how to tackle them
Outline
I will discuss the following unique aspects of our DevOps stack:
- Requirement of bringing up > 100 node hadoop clusters within 2 mins.
- Rolling deploys of Hadoop clusters and clients in the presence of long running jobs
- Autoscaling clusters to ensure optimal efficiency and expenditure
- Supporting multiple public clouds - AWS, GCE, Azure
- Allowing users to install their own software.
I’d like to discuss our strategies in handling issues like this and talk about tools that we’ve developed/discovered along the way.
Speaker bio
Currently working as a DevOps engineer at Qubole. Before this was a Platform engineer at BrowserStack, working on AWS and a custom private cloud setup. You can find more about me on LinkedIn - https://in.linkedin.com/in/umangdhawan
{{ gettext('Login to leave a comment') }}
{{ gettext('Post a comment…') }}{{ errorMsg }}
{{ gettext('No comments posted yet') }}