The Fifth Elephant 2015

A conference on data, machine learning, and distributed and parallel computing

HawkEye: A Real-Time Anomaly Detection System

Submitted by Satnam Singh, PhD (@satnam-datageek) on Monday, 8 June 2015

videocam_off

Technical level

Beginner

Section

Crisp Talk

Status

Confirmed & Scheduled

View proposal in schedule

Vote on this proposal

Login to vote

Total votes:  +9

Objective

In this talk, I will present the details of the HawkEye system with insights on selection of algorithms and parameter tuning. I intend to share our mistakes and learnings while deveoloping HawkEye.

Description

HawkEye is a real-time anomaly detection framework for detecting anomalies in IT infrastructure data e.g. CPU, memory, response time of a data center machine. The framework uses a combination of anomaly detection techniques to detect local and global anomalies. The system detects several types of local anomalies using different anomaly detection techniques over a sliding window. We used rigorous data experiments to perform data preprocessing and select hyperparameters. We have employed statistics-based techniques to detect local point anomalies and statistical detection theory (Page’s Test) to detect local contextual anomalies. Time-series models are employed to detect seasonality in the data and detect global anomalies.
In this talk, I will showcase results of the HawkEye system on real-data.

Requirements

Basic knowledge of statistics, machine learning, data mining will be helpful in understanding the talk.

Speaker bio

https://in.linkedin.com/in/satnamsinghdatascientist

Slides

http://www.slideshare.net/satnam74/hawkeye-a-realtime-anomaly-detection-system

Comments

Login with Twitter or Google to leave a comment