The Fifth Elephant 2012

Finding the elephant in the data.

A Herd of Elephants - Navigating the Hadoop Ecosystem

Submitted by Vinayak Hegde (@vin) on Wednesday, 25 April 2012

Section: Big Data Infrastructure & Processing Technical level: Intermediate Session type: Tutorial

View proposal in schedule

Abstract

A overview of the Hadoop ecosystem and how the different parts of the ecosystem interact and fit together.

Outline

Hadoop has matured to point where it is not longer just one project but a bunch of projects ranging from getting data onto the cluster to processing and analyzing data to managing the cluster itself. I will be talking from my personal experiences from setting up a hadoop cluster at Inmobi that processes 10TB+ of Data per day (and growing). The several Hadoop clusters in Inmobi are spread over multiple datacenters across continents.

Projects that will be covered in some detail include Hadoop (HDFS and Mapreduce), Hive, HBase, Pig, Mahout, Scribe , Zookeeper and Oozie/Azkhaban.

Requirements

Should have basic familiarity with Hadoop.

Speaker bio

Vinayak Hegde is Head of Engineering (Marketplace Management) at Inmobi. He has been active in opensource software community for more than a decade. He has been writing code in mainstream as well as esoteric programming languages on a variety of operating systems. He is a computer networking and data geek.

Comments

Login with Twitter or Google to leave a comment