Context: Early Startup

While a plethora of MLOps work has been done at large data and model serving scale, this is an attempt to introduce you to different ops problems, tools and how to pick among them. In most cases, I’ll share an example to anchor the idea set further.

Based on my experiences at Verloop.io, growing from 0 to 1. Verloop is among India’s largest conversational automation platforms. We work with some of India’s most largest category defining companies to serve their users.
Talk Objective

This is a primer on how to think about your experimentation/dev and deployment process. The bulk of this talk is organised in 2 columns: problem and a recommended process or tool around that. I borrow from my experience working on ML engineering challenges e.g. keeping latencies low enough for chat to be usable, things we learnt along the way e.g. need for data versioning and so on.

Slides for the Talk : https://bit.ly/startupmlops2021

Talk Outline

• MLOps Primer 101: Intro to layout of this talk [2-3 minutes]
  - Assumptions/Prerequisites: Team size, Skills, Organisation, Shipping [4-5 minutes
  - Level 1: DevOps: Builds, Tests and CI/CD [5-7 minutes]

• Intent: You don’t repeat a mistake which you’ve made previously

• Code Review: Github with the ReviewNB App

• Setup testing for business logic and app at the very least. Add them to CI/CD

• [Recommend] Test your ML models with automated tests like CheckList

• Level 2 MLOps: Experiments and Dev Cycle [8-10 minutes]

• Intent: Automated Training

• Our 3 key elements: data, code/architecture, model weights

• Data needs to be mise en place - ideally pipelines and ETL processes are figured out

• Reproducible pipelines -- focus on integrating with your existing infra, and it’s okay to simply pull using a Jupyter notebook
  It’s also ok to aim for more complexity, by setting up DAG tools like Airflow and ML FLow

• Experiment Tracking: Sacred, Neptune.ai, W&B

• Both training code and resulting models are version controlled

• DVC.org for Data Version Control

• Manual release- Don’t bother automating it so early

• Manage your own releases, add SDE to your team if needed

• [Optional] Managed compute

• Excellent time to adopt Docker and if your company is on K8, port your service to K8 as well

• Level 3 MLOps: Model Deployment [Total: 5-7 minutes]

• Intent: Automated Model Deployment

• Data pipeline gathers data without engineering time - including annotations at predictable cadence

• Release engineering: Adopt what your engineering team is doing: Canary or Blue-Green deployment

• Setup infra to A/B test your models or blends of them in shadow mode

• Concept, Vocabulary, Label Drift: Don’t bother. Your data pipeline will update your models for you

• Questions from Audience [3-5 minutes]

This is a process/maturity example and recommendation of when you should solve for a specific problem within MLOps. It’s not a hard and fast guide and you can always solve some problems sooner or later.

I use the Microsoft MLOps Model as a reference.

Some experience with the following would be useful for this talk:

• Working in a team for 2-10 data folks, across modeling, engineering, monitoring, deployment and so on
• Deploying models with specific requirements, e.g. low latency, high throughput, large data volumes at inference (more than 1TB)

Speaker Info:

Nirant has worked across startups and MNCs in Machine Learning and Data Science roles. These include:

• Verloop - Natural Language Process - Conversational Automation for Enterprises
• Soroco - Computer Vision: Image Segmentation - building Search for Enterprise Documents
• Samsung Research at the Advanced Technologies Lab - Senor Fusion & Event Classification
• Belong.co (NLP/Predictive Analytics)

At his present role in Verloop.io, he focused on Conversational AI

He has written a book on Practical NLP for Developers (Published by Packt). This book is a Quickstart Guide for Developers interested in building NLP based solutions, without the patience for pedantic learning on Linguistics and Deep Learning.

Recognition & Contributions

• Won the Kaggle NLP Kernel Prize from Kaggle and Explosion.AI (makers of spacy.io)
  Lead Maintainer for awesome-nlp with ~11K+ stars - recommended by Andrew Ng’s Deep Learning course CS229 at Stanford
• GitHub’s official Machine Learning collection includes awesome-nlp as world’s best NLP resource
• FastAI International Fellowship: 2018 & 2019

Talks

• PyCon India 2019: http://bit.ly/pycon2019talk (Google Slides Talk)
• inMobi Tech Talks: A Nightmare on the LM Street; Slides
• Wingify DevFest: NLP for Indian Languages; Slides, Video
• PyData Bengaluru Inaugural Talk: Video, Resources

Speaker Links

Personal Website: https://nirantk.com
Twitter: https://twitter.com/NirantK/
Github: https://github.com/NirantK
LinkedIn: https://linkedin.com/in/nirant
Book: https://www.amazon.in/dp/B07L3PLQS1/