Elastic Search users' meetup

A meetup for and by users of Elastic Search

Tickets

Elasticsearch as consistent Lookupstore

Submitted by Abhishek Kumar Gupta (@greedy93) on Monday, 1 July 2019

Abstract

Elasticsearch is a distributed system which is highly scalable, performant and provides near-real time search capability. But elasticsearch don’t have a concept of transactions and is infamous for losing data or serving stale data. Can Elasticsearch be used as a consistent lookup data store, where we can ensure that any write that has happened is available for all future lookups? Can we ensure that we always gets the latest document for a search request?

Outline

Agenda:

  • What is lookupstore?
  • Consistency & resiliency issues in ES
  • How are we trying to make it consistent
  • Trade offs of the approach.
  • Is the solution enough?

Requirements

Go through this doc for more on what ES team is doing - https://www.elastic.co/blog/resiliency-elasticsearch

Speaker bio

I am SDE 2 in Flipkart and am a part of Logistics Insights team building data streaming pipeline to enable the data driven decisions.

Comments

  • Denis Dsouza (@denisdsouza) 2 months ago (edited 2 months ago)

    Hi Abhishek Kumar Gupta,
    Just wanted to know your organizations use cases for Elasticsearch ?

    • Abhishek Kumar Gupta (@greedy93) Proposer 2 months ago

      Hi Denis,

      Our team is using elasticsearch to store data as both snapshot and incremental view and then use this store for lookups in data streaming pipeline. We are also having Kibana dashboards for monitoring purpose. There could be many other usecases for which ES is being used by other teams.

      Thanks

Login with Twitter or Google to leave a comment