Rootconf Hyderabad edition

At Nike Tech, we have a diverse technology landscape with teams working on building apps for the end consumer to dashboards for our internal business stakeholders. All this is done at massive scale and a strong focus on CI/CD.
CI/CD in data engineering is a often neglected area and weak practices leads to inefficiencies which hurt in the long run. With a focus on delivering highly trusted and consistent data to business in order to make decisions, the development and testing process needs to be watertight before things move to production.
CI/CD is also usually an afterthought when we are designing and building the data engineering applications. We here at Nike do it differently and ensure CI/CD is a key focus area from the very beginning and design accordingly. This talk throws light into our thinking process when it comes to CI/CD, the tools we use and design examples for a couple of use cases.

Outline

•What is CI/CD?
•Why CI/CD in data engineering?
•How to simplify CI / CD for Airflow using Jenkins and Code Repos like Git/Bitbucket
•How to simplify CI / CD for Nifi using Jenkins/Nifi Registry/Terraform

Speaker bio

Ram is a lead engineer at Nike Enterprise data and analytics organization working in the space of real time master data management, data ingestion and data science engineering.

Links

• https://www.linkedin.com/in/ramanareshchalla/

Slides

https://drive.google.com/file/d/1dI2fsehxJvI3s4IGGnv84ShNrAnCN9KJ/view?usp=sharing