Introduction and setup [~ 5 mins]

The aim of this section is to provide a gentle introduction to the system being examined. The use-case, requirements and setup of the tools will be discussed to benefit those not familiar with the intricacies of the domain.

topics covered:

basic overview of Postgres/SQL database management
solutions for concurrency control via transaction IDs
caveats to these solutions and its consequences

Deep dive into the story [~ 20 mins]

This section will form the crux of the talk. Beginning with investigations into reports of degraded performance, the story leads towards specific limitations of Postgres’s MVCC implementation.

Tools used in investigating the incident and various attempted solutions will be discussed. The focus will be on diagnosing/fixing systems without compromising on underlying performance and uptime requirements.

topics covered:

overview of use-case, system setup and configuration specifics
initial monitoring and tracking system performance reports
tools used to investigate the degradation
first attempt at a fix, its failure to correct, and subsequent learning
second attempt to brute-force a solution, and the resulting (failed) inconsistent state
final attempt at a simpler solution, and learning

Lessons, Upgrades and Conclusion [~ 10 mins]

The last section will cover lessons learned - including checks and systems that could have avoided the problem. The various knobs available to DevOps engineers to manage the performance of their systems will be explored. Combining personal experiences with industry best practices, the aim is to deliver concrete steps to simplify database management. Also, in the months since the incident, patches in newer Postgres releases which fix the problem will be listed.

topics covered:

knobs to turns and levers to pull when tuning systems
best practices to simplify Postgres management roles
updates and relevant patches available via newer releases

Rootconf 2017

Rootconf 2017