MLOps Conference
The Fifth Elephant For members

MLOps Conference

On DataOps, productionizing ML models, and running experiments at scale.

Make a submission

Accepting submissions till 14 Jul 2021, 11:00 PM

Tickets

Loading…

Machine Learning (ML) is at the helm of products. As products evolve with time, so is the necessity for ML to evolve. In 2010s, we saw DevOps culture take the forefront for engineering teams. 2020s will be all about MLOps.

MLOps stands for Machine Learning Operations. MLOps mainly focuses on workflows, thought processes and tools that are used in creating ML models, and their evolution over time. The workflows for ML at organizations are different as the problem space, maturity of teams and experience with ML tools are widely different.

MLOps relies on DataOps. DataOps is about Data operations, and helps define data and SLOs for data - how they are stored, managed and mutate over time - thereby providing the foundations for sound ML. The success and failure of ML models depends heavily on DataOps, where data is well-managed and brought into the system in a well thought out manner. ML and data processes have to evolve to provide insights into the reasons as to why certain models are not behaving as before.

Productionizing ML models is a challenge, but so is running experiments at scale. MLOps caters not only to scaling ML models in production, but also helps in providing guidelines and thought processes to support rapid prototyping and research for ML teams.

MLOps Conference 2021 edition

The 2021 edition is curated by Nischal HP, Director of Data at Scoutbee.

The conference covers the following themes:

  1. Machine Learning Operations
  2. Machine Learning in Production
  3. Privacy and Security in Machine Learning
  4. Tooling and frameworks in Machine Learning
  5. Economies of Machine Learning

Speakers from Doordash, Twilio, Scribble Data, Microsoft Research Labs India, Freshworks, Aampe, Myntra, Farfetch and other organizations will share their experiences and insights on the above topics.

Schedule: https://hasgeek.com/fifthelephant/mlops-conference/schedule

Who should participate in MLOps conference?

  1. Data/MLOps engineers who want to learn about state-of-the-art tools and techniques.
  2. Data scientists who want a deeper understanding of model deployment/governance.
  3. Architects who are building ML workflows that scale.
  4. Tech founders who are building products that require ML or building developer productivity products for ML.
  5. Product managers, who are seeking to learn about the process of building ML products.
  6. Directors, VPs and senior tech leadership who are building ML teams.

Contact information: Join The Fifth Elephant Telegram group on https://t.me/fifthel or follow @fifthel on Twitter. For inquiries, contact The Fifth Elephant on fifthelephant.editorial@hasgeek.com or call 7676332020.

Hosted by

The Fifth Elephant - known as one of the best data science and Machine Learning conference in Asia - has transitioned into a year-round forum for conversations about data and ML engineering; data science in production; data security and privacy practices. more

Supported by

Scribble Data builds feature stores for data science teams that are serious about putting models (ML, or even sub-ML) into production. The ability to systematically transform data is the single biggest determinant of how well these models do. Scribble Data streamlines the feature engineering proces… more

Promoted

Deep dives into privacy and security, and understanding needs of the Indian tech ecosystem through guides, research, collaboration, events and conferences. Sponsors: Privacy Mode’s programmes are sponsored by: more

swaroopch

@swaroopch

Maintaining Machine Learning Model Accuracy Through Monitoring

Submitted Jun 15, 2021

Machine Learning models begin to lose accuracy as soon as they are put into production. At DoorDash, we implemented a robust monitoring system to diagnose this issue and maintain the accuracy of our forecasts.

DoorDash is a last-mile logistics platform. We use Machine Learning to improve the quality of the experience of our customers. For example, reliable estimates for how long it takes for a restaurant to prepare a food order ensures that when a Dasher (our term for delivery drivers) arrives at a restaurant, the food is already ready, this leads to (a) less queueing at the restaurant by Dashers, leading to a positive experience for restaurants and Dashers both, (b) maximizes the earning potential for Dashers per hour, (c) minimizes the time for delivery to the consumer.

However, once an ML model is trained, validated, and deployed to production, it immediately begins degrading, a process called model drift. This degradation negatively impacts the accuracy of our time estimates and other ML model outputs. Because ML models are derived from data patterns, their inputs and outputs need to be closely monitored in order to diagnose and prevent model drift. Systematically measuring performance against real-world data lets us gauge the extent of model drift.

In the past, we’ve seen instances where our models became out-of-date and began making incorrect predictions. These problems impacted the business and customer experience negatively and forced the engineering team to spend a lot of effort investigating and fixing them. Finding this kind of model drift took a long time because we did not have a way to monitor for it.

This experience inspired us to build a solution on top of our ML platform. We set out to solve this model drift problem more generally and avoid issues like it in the future for all of the ML use cases on our platform. Ultimately, our goal was to create a solution that would protect all the different ML models DoorDash had in production.

In this talk, we will walk through our story of how we surveyed our data scientists, applied systems thinking to this problem, came up with an approach, and implemented a platform-level solution to preventing model drift.

We hope this story will be useful for other Machine Learning Platform teams who want to prevent issues such as data drift and model drift.

Full Article : https://doordash.engineering/2021/05/20/monitor-machine-learning-model-drift/

Slides : https://swaroopch.com/ml-model-monitoring

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Make a submission

Accepting submissions till 14 Jul 2021, 11:00 PM

Hosted by

The Fifth Elephant - known as one of the best data science and Machine Learning conference in Asia - has transitioned into a year-round forum for conversations about data and ML engineering; data science in production; data security and privacy practices. more

Supported by

Scribble Data builds feature stores for data science teams that are serious about putting models (ML, or even sub-ML) into production. The ability to systematically transform data is the single biggest determinant of how well these models do. Scribble Data streamlines the feature engineering proces… more

Promoted

Deep dives into privacy and security, and understanding needs of the Indian tech ecosystem through guides, research, collaboration, events and conferences. Sponsors: Privacy Mode’s programmes are sponsored by: more