Rootconf Mini 2024 (on 22nd & 23rd Nov)

Geeking out on systems and security since 2012

Raj Suvariya

@rajsuvariya

Maintenance operator for TiDB running on Kubernetes

Submitted Oct 30, 2024

Description:

TiDB is distributed SQL and horizontally scalable datastore developed by PingCAP. TiDB makes it easy to deploy and run database clusters on kubernetes by providing a official TiDB operator. However, this operator generally assumes that the database is running on network attached disk, which makes it easier for operator to not worry about compute failures - as compute can be rescheduled to any other compute server and disk failutes - as majority of NAS comes with redundancy at disk level.

However, working at Flipkart we realised that most of our workloads were running on local attached disks, which makes it difficult to perform any planned downtime / destructive maintenances at hardware leyar, considering two major problems, (a) compute cannot be rescheduled as it’s locally attached with disk (b) disk doesn’t have redundancy. To overcome this problem we developed another in-house operator which does end to end handling of proactive move out before the scheduled maintenance.

Talk will be covering lot of content present on this blog post written by me. https://blog.flipkart.tech/automating-tidb-planned-maintenance-with-k8s-operator-5893f3ffb657

Key takeaways

  • Running databases on kubernetes and it’s challenges wrt to different kind of hardwares.
  • Development of kubernetes operator which works side by side with another operator, both operating on same resource.

Target Audience

  • System engineers working on developing databases on kubernetes

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hosted by

We care about site reliability, cloud costs, security and data privacy

Supported by

Platinum Sponsor

Nutanix is a global leader in cloud software, offering organizations a single platform for running apps and data across clouds.

Platinum Sponsor

PhonePe was founded in December 2015 and has emerged as India’s largest payments app, enabling digital inclusion for consumers and merchants alike.

Silver Sponsor

The next-gen analytics engine for heavy workloads.

Sponsor

Community sponsor

Peak XV Partners (formerly Sequoia Capital India & SEA) is a leading venture capital firm investing across India, Southeast Asia and beyond.

Venue host - Rootconf workshops

Thoughtworks is a pioneering global technology consultancy, leading the charge in custom software development and technology innovation.

Community Partner

FOSS United is a non-profit foundation that aims at promoting and strengthening the Free and Open Source Software (FOSS) ecosystem in India. more

Community Partner

A community of Rust language contributors and end-users from Bangalore. We have presence on the following telegram channels https://t.me/RustIndia https://t.me/fpncr LinkedIn: https://www.linkedin.com/company/rust-india/ Twitter (not updated frequently): https://twitter.com/rustlangin more