Rootconf Delhi edition

On network engineering, infrastructure automation and DevOps


Strongly consistent centralized Document Service with eventual consistent Search Store

Submitted by Aman Bansal (@aman-bansal) on Saturday, 26 October 2019

Section: Crisp talk (20 mins) Category: Distributed systems Status: Rejected


Organizations with multiple teams generally end up with different applications that manage it’s own database interactions. This approach would be fine at a smaller scale but as we move forward towards a large scale, the situation will become more error-prone. Because now each application needs to either deploy experts in each team or build expertise to make the data available, consistent and usable at all times. This leads to wastage of resources both in terms of computing as well as human. In addition to that, applications need to manage Disaster Recovery (DR), migrations (if needed), Recovery Time Objective (RTO) and Recovery Point Objective which all leads to inconsistencies. This talk is about our experience of how we solved this problem at Mindtickle. We developed a strongly consistent centralised document service and we believe it is worth sharing with the community.



some development experience in client server applications

Speaker bio

Aman Bansal is a technology enthusiast who is lucky enough to hack on various projects at Mindtickle, Pune. Currently, he is part of the Infrastructure team which is developing a centralized database service solution.



  •   Zainab Bawa (@zainabbawa) Reviewer 5 months ago

    Thank you for the submission, Aman. Here are some questions, based on the details you have submitted:

    1. The context is unclear in the sense of what is the business that Mindtickle is in, what is/are the specific applications you are referring to (in this case study), and key metrics such as (usage and therefore data models and DR).
    2. Also, why was the need for a centralized document service ‘felt’? What was the system prior to this? What were the proposed solutions/approaches before it was decided that centralized document services is the way to go forward? Why does your business use case necessitate this approach?
    3. Similarly, what other approaches were considered and compared before deciding on the chosen approach? Share details.
    4. The before-after scenario has to be made clear with metrics, i.e., how was the situation before the proposed solution and what was the situation after implementing the proposed solution? What was compromised after implementation? What is the one big win or innovation of your proposed solution?
    5. Since the context and the application are unclear, the takeaways that you mention seem abstract and generic. This will need to be clarified (and also specified whether the takeaways extend to organizations that don’t have the same business use case as Mindtickle).

    Look forward to your responses.

    •   Aman Bansal (@aman-bansal) Proposer 4 months ago

      @zainabbawa Thanks for your review. Your thoughtful review and response is highly appreciated. And I would like to apologize first for the late response but I was occupied with some unavoidable circumstances. I have prepared this document with my response and I hope this will answer all your queries. Do let me know if you have anymore doubts.

      •   Zainab Bawa (@zainabbawa) Reviewer 4 months ago

        Thank you for the responses, Aman. In your response, you mention: ” we @Mindtickle believe the community no matter what use cases they have is in need of a solution that can mitigate the issues I have mentioned in the previous document.”
        Give us 1-2 examples of how this issue can be applied to a business use case other than Mindtickle’s, hypothetically, if not real.

        •   Aman Bansal (@aman-bansal) Proposer 4 months ago (edited 4 months ago)

          Hi @zainabbawa. The proposed solution can have many use cases across different organizations. But to name a few, we think organizations having following use cases, would be benefitted from our approach:

          1. DB Agnostic APIs: It might happen in an organization where different teams are working on different applications and they have different DB needs like one application needs a relational database, or others might need document storage like Couchbase or might have different search needs for elastic search. Our proposed solution exposes DB agnostic API’s where intended DB needs are controlled via configurations. The proposed solution is capable to handle all these needs and should attract the attention of organizations looking for deploying DB agnostic database service.

          2. Multitenancy use case: This use case is more specific to SAAS based organization where they handle data for different tenants. The proposed solution provides more support and control over multiple tenants in terms of data separation, security, and control. This particular feature was not available in other solutions like Vitess and CockroachDB. They work on a principle where there could be multiple tenants data present in one single shard.

          3. Centralized data governance use case: The proposed solution empowers the organization in terms of deploying specialized teams for governing the database operations like Backup, monitoring, alerting, auditing of data, security, etc. Because of centralized service for all the applications, now there is no need for scattered expertise among different teams to manage the above operations. This leads to better management of resources and the optimal functioning of business operations.

          I hope the above points will answer your doubts. But do let me know if there is anything I can help with.

          •   Anand Chitipothu (@anandology) 4 months ago

            Hi @aman-bansal, it looks like you are solving an interesting problem. I’ve some questions about the proposal and the above comment.

            1. Could you please explain a bit more about the “DB Agnostic APIs”? Does it mean, the service can speak SQL like a relational DB, another API like a document store and yet another for search? What all features of SQL can it support? Very often, abstractions like this comes with a performance penalty and many limitations. I don’t think the proposal is complete without mentioning them.

            2. Adding multi-tenancy support to a SQL database doesn’t look like a very difficult problem. Have you considered adding a wrapper to something like CockroachDB to provide multi-tenancy support? What were the reasons to think that that would not meet your requirements?

            3. Other than centralized governance, I don’t understand why a centralized database system is a better idea. You lose the isolation between projects. It also comes at a cost of taking away the flexibility from the teams. The teams are forced to work with a single database system irrespective of the data size and complexity.

            •   Zainab Bawa (@zainabbawa) Reviewer 4 months ago

              Thank you for the comments, Anand. Aman, your responses?

  •   Zainab Bawa (@zainabbawa) Reviewer 4 months ago

    Hello Aman,

    We have reviewed your proposal. The key concern is that your proposal is too focussed on an organization’s use case, thereby making it narrow and less extensible to what participants who don’t have similar org concerns and challenges. We are unable to accept your proposal for the conference given the narrowness and limitations in scope.

    All the best for taking this work forward. We’ll look forward to the progress on this project, and if it fits future editions of Rootconf as your learnings and insights grow.

Login with Twitter or Google to leave a comment