The Fifth Elephant 2019

Taking deep learning models to production, and doing so reliably, is one of the next frontiers of DevOps. This talk introduces RedisAI, a joint effort by [tensor]werk and RedisLabs. RedisAI is a Redis module that adds tensors & graphs as Redis data types, enabling execution of deep learning graphs on the CPU and GPU using multiple backends (PyTorch, TensorFlow, and ONNXRuntime) simultaneously, while exposing a full tensor API for scripting. In this talk, we will demonstrate deploying a deep learning model to production in a highly available environment and lay down the roadmap towards 1.0.

Outline

Year 2018 was the year of model servers. There were numeroius initiatives for building a reliable, interoperable deep learning deployment toolkits but so far we don’t have an easy tool that can reliably handle the deep learning models from all the frameworks. With the advent of Redis modules and the availability of C APIs for the major deep learning frameworks, it is now possible to turn Redis into a reliable runtime for deep learning workloads, providing a simple solution for a model serving microservice. In this talk we will introduce RedisAI, a joint effort by [tensor]werk and RedisLabs that introduces tensors and graphs as new Redis data types and allows to execute graphs over tensors using multiple backends (PyTorch, TensorFlow, and ONNXRuntime), both on the CPU and GPU. The module also supports scripting with TorchScript, which provides a Python-like tensor language that can be used to facilitate pre- and post-processing operations, like input shaping or output ensembling. In addition, thanks to its support for the ONNX standard, including ONNX-ML, RedisAI is not strictly limited to deep learning, but it offers support for general machine learning algorithms. In this talk, we will demonstrate a full journey from training a model to deploying to production in a highly available environment. Last, we will lay down the roadmap for the future, like automated batching, sharding, integration with Redis data types (e.g. streams) and advanced monitoring. The talk will include sample code, best practices and a live demo.

##Who should attend this tutorial:

• Leads who are managing a DL, ML, and traditional ML teams
• DL/ML engineers
• DL/ML researchers who needs to to interact with the engineering team for production deployment
• DevOps
• If you plan to do production deployment of DL/ML/traditional ML products

Requirements

Participants attending this tutorial must install the following software before attending the session:

1. Laptops with Linux or Mac OS installed. If you have a Windows machine, set up a cloud instance.
2. Docker
3. Python 3.6+
4. pip install PIL
5. pip install Numpy
6. pip install redisai==0.3.0
7. pip install mlut[all]
8. Jupyter notebook

Speaker bio

I am working as a part of the development team of tensorwerk, an infrastructure development company focusing on deep learning deployment problems. I and my team focus on building open source tools for setting up a seamless deep learning workflow. I have been programming since 2012 and started using python since 2014 and moved to deep learning in 2015. I am an open source enthusiast and I spend most of my research time on improving interpretability of AI models using TuringNetwork. I am part of the core development team of Hangar and RedisAI and a constant contributor to PyTorch source. I also have authored a deep learning book. I go by hhsecond on internet

Links

• https://medium.com/tensorwerk/redisai-thors-stormbreaker-for-deep-learning-deployment-a3fe7569c559

Slides

https://docs.google.com/presentation/d/1R_HrFWayzKEpom24zVfSF0VGbOfBgLyR0XTl1hNxT9E/edit?usp=sharing