PyCon Pune 2017

From constant monitoring of data using Excel and analyzing countless graphs to developing increasingly complex statistical models to automate the whole procedure, IoT has come a long way in its approach to make sense of high velocity streaming data. But to realize their true potential, applications must be able to predict well in advance what failures are likely to occur and how to mitigate them. This is where anomaly detection comes into picture.

Ever wondered why a cardiologist spends hours looking a patient’s medical history? Or why banks keep an extremely detailed track of their users’ transactions? Time series data provides an answer to how your regular processes should behave and why in some bizarre cases they do not. Fault detection and prediction is essential for the continuous operation of devices.

This talk will be about how machine learning algorithms are a boon to this industry where, even one second of a deviation of 10^(-5) in one variable can result in your system being shut down. The explosion of Big Data already has everyone running to use the state-of-the-art techniques to manage this data. To make sense of it, one must return to the basics - sliding window and moving averages and move towards the more capable - feature extraction, supervised and unsupervised learning.

I will explain why some rudimentary algorithms fail in case of multivariate time series data and why there is a need to introduce feature selection and machine learning in order to employ a more insightful approach. This talk will also cover how these algorithms are used by various other industry sectors as well as in solving some of the world’s more sophisticated research problems like handwriting and speech recognition. In specific, I will explain how the a simple IoT application can benefit from these data science techniques in order to give their processes and products an edge over the others.

Outline

• Intro to Anomaly Detection and IoT
• Applications
• Time series analysis
• Algorithms currently in use
• ML algorithms in use
• Python libraries used
• Code demo
  • EDA: Basic techniques
  • Fault detection

Requirements

• Preliminary knowledge of descriptive stats, ML and Python
• Love for Pandas (the library)

Speaker bio

A Math nerd, I love playing with numbers, be it the stats of a match scoreboard or the digits on a vehicle number plate. Math and IT graduate, I currently work at Ecozen, an IoT company, as a data scientist. Passionate about cricket, so started with Python a couple years back with cricket analytics, modelling the batting order of an ODI match lineup.
Can’t live without Pandas. ML enthusiast and a big fan of social entrepreneurship.

Links

• Get an introductory idea from: https://slides.com/shreyakhurana/anomaly-detection
• IPython notebook: https://github.com/ShreyaKhurana/MuPy/blob/master/MuPy_Anomaly_Detection.ipynb
• LinkedIn: https://www.linkedin.com/in/shreya-khurana-36759a86
• GitHub: https://github.com/ShreyaKhurana