Scaling ElasticSearch for Log and Event monitoring
Submitted by Arpit Mohan (@mohanarpit) on Tuesday, 21 January 2014
Tracking application logs and events using ElasticSearch thereby giving a fast and simple way to query them and generate useful metrics.
In any application, logs and events form the basis of measuring, debugging and monitoring the application. Tracking events also allows businesses to measure the impact of features, new deployments and closely keep an eye on numbers that matter and take decisions based on these metrics.
There are many great tools for working with either logs or metrics (StatsD, Loggly etc). Unfortunately, most tools out there focus on only one of them. ElasticSearch is a generic (and very scalable) tool which allows developers to not only store and parse logs but also store rich event data. This event data is then used to create usable metrics.
In this talk, I aim to highlight how to leverage and scale an ElasticSearch cluster for the purposes of log/event tracking.
Arpit Mohan - CTO, Clinknow
A technology enthusiast with a penchant for leveraging technology to solve hard problems, he started his foray into the startup world with Gharpay. Currently, he leads the technology efforts at Clinknow, a BigData initiative with a focus on financial analysis.