The Fifth Elephant 2012

Finding the elephant in the data.

Crunching big data, Google scale

Submitted by Rahul Kulkarni on Monday, 9 July 2012

videocam_off

Technical level

Beginner

Section

Big Data Infrastructure & Processing

Session type

Lecture

Status

Confirmed

Vote on this proposal

Login to vote

Total votes:  +15

Objective

Insights from Google's experience on handling big data, overview of techniques and products (with case studies) on crunching big data.

Description

Whether you are an e-commerce startup or a genomics research lab or plan to run products at the scale of Gmail, Youtube or Adwords, you generate gigabytes of data every day if not every hour. Your storage requirements run into tera or peta bytes and you may need thousands if not hundreds of thousands of CPU cores to process that data. At Google, we have developed several in-house tools and techniques to be able to process data at scale. Recently we made several of these tools available externally. In this talk we will go over some of our learnings on big data, and discuss techniques with case studies for crunching big data.

Speaker bio

Rahul Kulkarni joined Google in 2006 as its first product manager in India. He currently manages a cross functional team leading Google maps and local efforts in India. He has built and led teams around cloud computing, Google Apps platform, Orkut, OpenSocial, Google Finance and ads at Google. Prior to joining Google, Rahul led new product development efforts at National Instruments Corp, Austin in the areas of design, prototyping and deployment of high speed control systems.

Comments

Login with Twitter or Google to leave a comment