Scaling databases with a database access layer

This submission has been added to the schedule

Scaling databases with a database access layer

Submitted Apr 26, 2014

Section: Full talk Technical level: Intermediate

The objective of this session is to discuss an architecture to scale and manage the database tier by introducing a database abstraction layer. We will discuss various challenges of database scalability and then discuss few solutions.

We will be taking few references from the Facebook’s WebscaleSQL project, Google’s spanner paper and MySQL proxy.

Outline

When an application goes viral it not an easy feet to scale the infrastructure to support it. It requires some crafty deployments, constant improvement in the application stack and a lot of automation in the infrastructure to serve the growing business.

For the past couple of years, I have been looking at the problem of scaling the database tier efficiently. Database (especially relational database) remains an interesting frontier to scale up in the multi-tier architecture. NoSQL offers promise of scale for certain use cases, but there a large ecosystem for SQL that remains and will continue to grow. I propose a database abstraction layer that offers great potential to scale, optimize, perform and manage the traditional database tier. This layer will virtualize the databases from applications and provides a single interface to DBAs, DevOps and Developers to manage, test and develop their database tier.

I will be talking about certain common optimization techniques that can be offered from this layer:

Instant scale up - transparent connection pooling and multiplexing, transparent caching, surge protection
Transparent scale out - read/write split, load balancing, query routing, Sharding
Automatic high availability - automatic failover
Real-time Analytics

Requirements

Inquisitiveness

Speaker bio

I am currently the VP of Engineering at ScaleArc, where we are building a solution like I described above. For the last 12 years, I have been working on building high-performance sub-systems on Linux in file systems, storage and networking domain. Most recently I was the founding engineer at CloudVelocity, where I helped build a hybrid cloud platform. Before that I worked as a Principal Software Engineer at Symantec, Cisco Systems and NeoPath Networks, and a contributor to OpenStack. The time I spent in these startups was a great experience and I hope I can share some of these learnings with fellow engineers.

In short, I enjoy sharing ideas on innovation, engineering and product development. I am looking forward for all the interesting sessions proposed on this conference.

Rootconf 2014