Make a submission

Accepting submissions till 28 May 2020, 09:30 PM

Online

1. What topics will be discussed?

  1. Experiences running systems at scale in production
  2. Fire stories

2. Who should attend this meetup
People interested in learning about distributed systems and who enjoy post-mortem/RCA’s.

3. Who should speak at this meetup?

  1. Have you been the primary respondent to a fire in production? We’d love to hear your story.
  2. Are you running systems at scale? We will be thrilled to hear experience reports of what went well.

5. Format of the meetup
Two talks, 40 minutes each.

Date (tentative): 30th May, 4-6 PM.

How to join the meetup: This event is free to attend. Zoom Link will be shared closer to the meetup date.

Contact details: write to Nishant Modak on modak.nishant+dsmeetup@gmail.com or team@dsconf.in or call +91-9823088236 Follow the meetup group on Twitter - twitter.com/dist_sys

Hosted by

A group for anyone interested in discussing building distributed systems. more

Goutham Veeramachaneni

@gouthamve

Consistent Hashing: Using something simple to scale infinitely and horizontally

Submitted May 26, 2020

We build and run https://cortexmetrics.io, a scalable time-series database that scales to billions of datapoints per day and 100s of Millions of active time-series. The architecture is modelled around Consistent Hashing, a (imo) simple but powerful algorithm to distribute data that lets you scale horizontally. We will first start with motivation (what are we trying to achieve), and introduce consistent hashing and how typical databases use Consistent Hashing with replication to scale. We will then talk about the limitations and challenges.

Finally we will talk about how we actually implemented it in https://cortexmetrics.io and https://github.com/grafana/loki to scale to 100s of millions of series and what outages were caused by it and how we improved on it, to give the attendees a practical takeaway.

NOTE: The meetup on 30 May is too short-notice for me. I would love to do a more prepared and thought out version of the talk in the next DS meetup though.

Outline

  1. Motivation -- what are we trying to achieve
  2. Consistent Hashing, what and brief history (Dynamo Paper)
  3. Consistent Hashing in Cassandra, how they use it to do replication and repair
  4. Consistent Hashing in Cortex and Loki (leveraging etcd to do it and how we architected it)
  5. Challenges and outages and future improvements planned
  6. Maybe even considerations if you want to do it yourself

Speaker bio

Goutham Veeramachaneni is a developer from India who started his journey as an infra intern at a large company where he worked on deploying Prometheus. After the initial encounter, he started contributing to Prometheus and interned with CoreOS, working on Prometheus’ new storage engine.
He is now an active contributor to the Prometheus eco-system and at one point the maintainer for TSDB, the storage engine behind Prometheus 2.0.
He works at Grafana Labs, on Cortex and open-source observability tools.

When not hacking away, he is on his bike adding miles and hurting his bum.

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Make a submission

Accepting submissions till 28 May 2020, 09:30 PM

Online

Hosted by

A group for anyone interested in discussing building distributed systems. more