The Fifth Elephant 2019

Background
Data is the new oil and its size is growing exponentially day by day. Most of the companies are leveraging data science capabilities extensively to affect business decisions, perform audits on ML patterns, decode faults in business logic, and more. They run large number of machine learning model to produce results.

Problem Statement
Managing ML models in production is non-trivial. The training, maintenance, deployment, monitoring, organization and documentation of machine learning (ML) models – in short model management – is a critical task in virtually all production ML use cases. Wrong model management decisions can lead to poor performance of a ML system and can result in high maintenance cost and less effective utilization. Below are the key concern for model management:

1. Computational challenges: machine learning model definition and validation, decisions on model retraining, adversarial settings.
2. Data management challenges: lack of a declarative abstraction for the whole ML pipeline, querying model metadata, model interpretation.
3. Engineering challenges: multiple tools and frameworks make integration complex, heterogeneous skill level of users, backwards compatibility of trained Models and hard to reproduce the training result.

Existing Solution
There are custom ML platform to address the above concerns such as FBLearner by Facebook and Michelangelo by Uber but they have their own limitations like:

1. They standardize the data preparation, training and deployment loop specific to particular platform and business needs.
2. They are limited to a few algorithms and frameworks.
3. They tied to one company infrastructure and hard to open source.

Why MLflow?
Databricks team found above concerns as their motivation to develop MLflow as an open source and cloud agnostic machine learning model management platform. Benefits of MLflow from machine learning model management:

1. Works with any ML library and language.
2. They are platform independent i.e. ML models run in same way anywhere example local system or any cloud platform.
3. Designed to be useful for 1 or 10000 person organisation.

Outline

Key focus area for Machine Learning Model Management with MLflow:

1. Managing ML models in production is non-trivial. What are the challenges and concerns of machine learning management lifecycle?
2. What is machine learning model management?
3. Motivation and concepts behind introduction of MLflow
4. How to solve problem of model management using MLflow?
5. MLflow components
6. Realtime problem and use case

Requirements

Basic understating of machine learning and its workflow

Speaker bio

Ravi Ranjan is working as Senior Data Scientist at Publicis Sapient. He is part of Centre of Excellence and responsible for building machine learning model at scale. He has worked on multiple engagements with clients mainly from Automobile, Banking, Retail and Insurance industry across geographies. In current role, he is working on Hyper-personalized recommendation system for Automobile industry focused on Machine Learning, Deep learning, Realtime data processing on large scale data using MLflow and Kubeflow.
He holds Bachelor degree in Computer Science with proficiency course in Reinforcement Learning from IISc, Bangalore.

Links

• 1. http://sites.computer.org/debull/A18dec/p5.pdf
• 2. https://www.parallelm.com/model-governance-needed-production-ml/
• 3. https://www.linkedin.com/in/ravi-ranjan-03/

Slides

https://drive.google.com/open?id=19fVbkGPGZrc973JVYOZvMCxaDjG78nIA