Anthill Inside 2019

On infrastructure for AI and ML: from managing training data to data storage, cloud strategy and costs of developing ML models

Tickets

Productionizing deep learning workflow with Hangar, <frameworkOfYourChoice> & RedisAI

Submitted by Sherin Thomas (@hhsecond) on Friday, 26 April 2019


Preview video

Technical level: Intermediate Section: Workshops Session type: Workshop

Abstract

Managing DL workflow is always a nightmare. Problems include handling the scale, efficient resource utilization, version controlling the data etc. With the heavily organized Hangar, we can keep the data on check now, not as a blob but as tensors in the data store and version at. The super flexible PyTorch gives us the advantage of prototyping faster and iterate smoother. The model prototype can now be pushed to RedisAI, the highly optimized production runtime, as torchscript and scale the serving to multi node redis cluster or redis sentinel, with high availablility ofcourse

Outline

Even with the advancement, the community had made in the past couple of years, problems of a DL engineer starts right from the beginning when they think about version controlling the data and model. None of the toolset available right now could make a platform “git” had provided for programmers years ago. With the entry of Hangar and it’s python APIs, we are now moving ahead of the game. Having a version controlling system like hangar in place, DL folks are still struggling with production deployment with their framework of choice. PyTorch is the most flexible and easy framework that all deep learning developers had loved. But without a plug n play deployment toolkit, PyTorch always suffered to attract people who want to deploy their model to production. Meanwhile, RedisAI is trying to solve the problem every deep learning engineer faces especially when they try to scale. Making sure the production environment is highly available is probably the most daunting task DevOps experts worried especially when they have a deep learning service in production. RedisAI, along with the integration to all other Redis tools in the ecosystem, is trying to bring super easy deployment platform for the user. In the talk, I’ll be presenting the killer combination of PyTorch and RedisAI and explain how can we develop super optimized DL model with LibTorch but without losing the flexibility provided by PyTorch and how to ship it to production without even worried about writing a wrapper Flask/Django service for the model. Not just that, I’ll be showing how to put this deployment into a multi-node cluster and make sure you have 100% availability always. In a nutshell, the talk covers a deep learning workflow by introducing three toolkits for the user that makes the whole pipeline seamless.

Requirements

The audience should have basic python knowledge and a brief understanding of deep learning.

Speaker bio

I am working as a part of the development team of tensorwerk, an infrastructure development company focusing on deep learning deployment problems. I and my team focus on building open source tools for setting up a seamless deep learning workflow. I have been programming since 2012 and started using python since 2014 and moved to deep learning in 2015. I am an open source enthusiast and I spend most of my research time on improving interpretability of AI models using TuringNetwork. I am part of the core development team of Hangar and RedisAI and a constant contributor to PyTorch source. I also have authored a deep learning book. I go by hhsecond on internet

Links

Slides

https://docs.google.com/presentation/d/1G7lIHEluM8SLdWiGp8As9Efcq-4GHAU5xXGHCZYvvFg/edit?usp=sharing

Preview video

https://youtu.be/urketowJWIQ

Comments

  • Abhishek Balaji (@booleanbalaji) Reviewer 3 months ago

    Hello Sherin,

    Thank you for submitting a proposal. We noticed you’ve submitted a workshop proposal as well. Do make a decision on how you want to structure your content. If you’re choosing to go with the talk, to proceed with evaluation, we need to see detailed slides and a preview video for your proposal. Your slides must cover the following:

    • Problem statement/context, which the audience can relate to and understand. The problem statement has to be a problem (based on this context) that can be generalized for all.
    • What were the tools/options available in the market to solve this problem? How did you evaluate alternatives, and what metrics did you use for the evaluation?
    • Why did you pick the option that you did?
    • Explain how the situation was before the solution you picked/built and how it changed after implementing the solution you picked and built? Show before-after scenario comparisons & metrics.
    • What compromises/trade-offs did you have to make in this process?
    • What is the one takeaway that you want participants to go back with at the end of this talk? What is it that participants should learn/be cautious about when solving similar problems?
    • Is the tool free/open-source? If not, what can the audience takeaway from the talk?

    We need to see the updated slides on or before 21 May in order to close the decision on your proposal. If we do not receive an update by 21 May we’ll move the proposal for consideration at a future event.

  • Sherin Thomas (@hhsecond) Proposer 3 months ago

    Hi Abhishek, Thank you for reviewing my submission. I had submitted a proposal for the workshop as well thinking productionizing deep learning is not a subject I could teach in 30 minutes. The idea was to give a thorough introduction to different toolsets in the talk and then take people through a hands-on session in the workshop if somebody shows interest. But unfortunately, I have deleted my workshop submission by mistake. Would be great if you can help me reopen it, but it is absolutely fine if that’s not possible. About the slides and videos, I’ll try to upload them before 21st.

    • Abhishek Balaji (@booleanbalaji) Reviewer 2 months ago

      Hey Sherin,

      No worries. We have not started evaluating workshops yet, so you can still submit one. Unfortunately, we’re unable to recover your deleted proposal.

  • Sherin Thomas (@hhsecond) Proposer 2 months ago

    Hi Abhishek, So I have made this talk as workshop talk and making another submission for RedisAI. I realized it’s never going to be a good talk if I include a lot into this 40 minute slot.

Login with Twitter or Google to leave a comment