Hands-on introduction to Pig arrow_forward
How big data moved the needle from monolithic SQL RDBMS to distributed NoSQL
Submitted by Sunil Sayyaparaju on Monday, 11 June 2012
Section: Big Data Infrastructure & Processing Technical level: Intermediate Session type: Lecture
The objective of this talk is to talk about different types of DBMS solutions (SQL/NoSQL - monolith/distributed) and the type of applications that are appropriate for a type of DBMS. The talk will highlight the design choices made by the developers for these different types of DBMS.
As we know, we are in an age of data explosion. More and more data started to pour in. New types of data like unstructured data also started to flow in. Different use-cases (like real-time queries) started to emerge. In this talk we will take a journey from the monolithic SQL RDBMS solution, to distributed shared-disk SQL, to distributed shared-nothing SQL, to distributed shared-nothing NoSQL solutions.
Along the journey we will see what factors contributed to the evolution of the next thing and what kind of design choices were made by the engineers along the evolution. We will also see what we got rid of (or the tradeoffs) during the evolution process. We will talk about what kind of applications will be best suited to a particular type of database.
I am in the database field for the past 8 years. I worked in internals of different types of SQL RDBMS solutions like single machine(monolithic), in-memory, distributed shared-disk, and distributed shared-nothing architectures. I worked primarily in transaction management, storage, access, performance tuning, and recovery areas.
I am currently working on distributed shared-nothing NoSQL solution(Citrusleaf). Citrusleaf is a high-performance, self-balancing, Immediately consistent, distributed NoSQL database. We developed an addon product for replication across data-centers over WAN which supports different complex topologies.