Lessons from Elasticsearch in production
Submitted by Swaroop (@swaroopch) on Wednesday, 11 June 2014
This talk is for people who are planning to use Elasticsearch in their next project.
Elasticsearch is one of the core pieces of infrastructure at Helpshift, a mobile-first CRM product.
We have had mistakes and lessons learned in production, and we would like to share that so that others have a smoother experience with Elasticsearch.
The customer-facing side of Helpshift product is a simple chat feature within the app using the Helpshift mobile SDK. The business-facing side is a complex agent dashboard that helps the agent in processing as many issues as quickly as possible. We will be focusing on this business-facing side, the designs we built on top of Elasticsearch and the problems we faced and how we went about solving them, a few of them are:
- Architecture 101 - Importance of separating master-only and data-only nodes, etc.
- How we index documents for each customer - the flaw in having one index per customer, and possible solutions
- Multilingual data - the importance of the phonetic plugin
- Complex views - the importance of understanding filters and how to combine them
- The importance of benchmarking - how we implemented using percolators for live notifications of new issues
- Restarts & Upgrades - the importance of disabling shard allocation and clustering
- Bulk Indexing - the importance of controlling replica count, etc.
- Runtime debugging - the importance of cat APIs, etc.
Preferably, audience has some basic understanding of what Elasticsearch does.
I work in the backend team at Helpshift.com, a customer service platform for mobile apps, our customers include Flipboard, Supercell, Flipkart, and several others.
I have previously worked at Automatic.com (the "UI for your car" company), Infibeam.com, Adobe and Yahoo!.
I have also written a couple of Creative Commons-licensed books - A Byte of Python and A Byte of Vim.