If you do not measure it, it does not exist! (Metrics with StatsD and Graphite)
Submitted by Rekha Joshi (@rekhajos) on Friday, 8 August 2014
If you cannot measure it, it does not exist!
If you do not measure it, it does not exist!
Before making a decision we need to measure, Before optimizing, we need to measure, Before investing we need to measure.
Whether it is cloud deployment, services or web application we always need the metrics. What we need to measure can range from network, machine or application specific metric. We want the metrics to tell us all we need to know about the product or service.
The world of front-end development with engaging visualization, great performance, and solving customer problem really well fails if not supported by good metric collection system. Metrics are the only reliable way to hear what your customers are saying.
The collection and consistency of the having metrics from our multiple products, services flowing into a single integrated dashboard has been our challenge.
This session intends to
1. Emphasize the importance of measurement.
2. How we need to do it all the time – from web, services, cloud
3. Demo measurements with StatsD and Graphite
This presentation will demonstrate how we have used StatsD with Graphite to have good metric collection framework.
The presentation would begin by introducing StatsD, a Node.js daemon which allows sending metrics over UDP. It would go into the implementation of StatsD and some of the core concepts of metrics, backend integration. We will touch upon the different language implementations of StatsD available in scala, python, ruby and C.
The talk will then introduce Graphite, the enterprise-scale monitoring tool that can take any time series data and render graphs. And highlight the integration of StatsD and Graphite.
With an example, the demo will show installation of StatsD and it listening for messages, parsing and extracting metrics data. This on integrated environment will get flushed to graphite. The Graphite carbon component takes in the time series data and displays it on graphite django webapp to render graphs.
Wrap Up: Steps to measure (for a five-year-old kid)
1. Install and Configure StatsD
2. Integrate with Graphite
3. Let the magic happen!
4. Boom! The Graphs give the Metrics details.
5. Thank node.js for making it so simple!
This cool capture and quick illustration of graphs enabled from StatsD and Graphite integration is what is making metric collection so easy.
There is now no excuse not to measure!
Rekha Joshi is a developer at Intuit and tries to listen what data and metrics say. Her role at Intuit involves solving challenges related to data.
Intuit works on multiple products and platforms ranging from accounting systems, payments solutions to cloud applications and services. It has metric collection framework based on Graphite, Riemann, Sensu and polls data over event bus. We are obsessed with what our data tells us.