Redis Miniconf

Realtime, user-facing statistics can really juice the user experience, especially for sites that are gamified or which implement complicated adaptive algorithms. Our test-prep app is both and I’ll be showing the stuff that went right and horribly wrong while building a realtime, user-facing statistics service for our web application. Hopefully, this experience will help you avoid many of the pitfalls and smoothen the journey to your own realtime statistics server.

There are a number of approaches to this problem and the right answer is very much dependent on your use case, so hopefully this talk will help you decide which way works better for you as well.

Also, if you’re wondering what all the fuss is about redis, do come for the talk. Redis can do so much more than caching and background tasks!

Outline

Tried building a decent BI/Analytics system lately? One that works in realtime? At scale?

In this talk, I’d like to share my experiences building a realtime stats server for our applications. We’re an EdTech company currently building a product that helps students prepare for highly competitive tests. Realtime feedback to users help us improve the user experience, improves learning outcomes and increases engagement. Being already in a competitive environment, students are enamoured of leaderboards and other such statistics. Making the system adaptive requires up-to-the-second information regarding the student’s grasp of the concepts, levels of fatigue, etc. Gamifying the learning experience properly depends on realtime information.

To do all of this, we needed a Real-Time Stats Service which was incredibly fun to build. This particular system takes the best of statsd and Storm and ends up with a system that is trivial to set up and monitor while still providing strong guarantees about once-and-only-once processing. My talk will present the use cases and business goals of the system, the challenges of building such a system, the tools available to build such a system and their limitations, some of the wrong turns I took (there were a few) and finally present a demonstration of a massively scalable, realtime statistics server, designed to give every stakeholder a clear picture of user data.

Speaker bio

I’m the CTO of Indiavidual Learning. We have a test preparation product that helps students prepare for highly competitive entrance exams (http://ultimate.embibe.com)

I’ve been developing web apps since the web was invented and I’m totally loving the Cambrian explosion in datastores that we’re witnessing at the moment. Over the last few months I have dived deep into more than a few of them and it’s really hard to find a better datastore than Redis for price, performance and just overall cool features!