Migrating from traditional EC2-like infra to Kubernetes

Submitted Apr 13, 2020

Duration of the session:: 40 mins full talk Category of talk: Kubernetes

This talk is inspired by our journey at Zapier to migrate our app infrastructure from traditional AWS EC2, terraform, Ansible based workflow to Kubernetes. The challenges we faced during this journey, the solutions, and finally the rewards. I strongly believe that this talk will help many people who are in a similar position we were when we started this journey. They want to use Kubernetes to speed up application deployments, pack applications densely, etc., but are skeptical to take the plunge from traditional EC2 based deployment models to Kubernetes.

Outline

State of things at Zapier, pre Kubernetes
Challenges faced
How Kubernetes promised to solve the problems?
Looking for a stable Kuberenetes distribution
Setting up Kuberenetes platform: cert-manager, external-dns, ingress, etc.
Manage Kubernetes artifacts
Testing the waters: running staging envs, non critical internal apps, etc.
Disaster management and migration in Kubernetes
How to load balance apps between previous infrastructure and Kuberenetes?
Start running Zapier monoliths in k8s to handle a little of prod traffic
Migrate role by role
90% of Zapier running on Kubernetes
Time to run stateful apps in Kubernetes
Always launch multiple Kuberentes clusters in production and load balance apps for HA
Monitoring
Where are my UDP packets?
Beware of noisy neighbours. Do proper resource planning and setting for apps.
Continuous deployment
Current state of things. Cost savings, deployment times, rollback times, etc.
More challenges. We love it
The future ahead

Requirements

Understanding of AWS EC2 like infrastructure and deployment models with Ansible, Terraform, etc.
Some knowledge about CI/CD
Basic understanding of Kubernetes

Speaker bio

Ratnadeep Debnath AKA rtnpro works as a Site Reliability Engineer at Zapier. He has deep interest in Kubernetes and CNCF projects. He loves contributing to Open Source and mentoring people to start their Open Source journey. When he is not coding, he likes spending time with friends and family, practise Mantra Meditation or cooking.

On Container Orchestration