The Fifth Elephant 2012

Finding the elephant in the data.

Vinayak Hegde

Vinayak Hegde

@vin

A Herd of Elephants - Navigating the Hadoop Ecosystem

Submitted Apr 26, 2012

A overview of the Hadoop ecosystem and how the different parts of the ecosystem interact and fit together.

Outline

Hadoop has matured to point where it is not longer just one project but a bunch of projects ranging from getting data onto the cluster to processing and analyzing data to managing the cluster itself. I will be talking from my personal experiences from setting up a hadoop cluster at Inmobi that processes 10TB+ of Data per day (and growing). The several Hadoop clusters in Inmobi are spread over multiple datacenters across continents.

Projects that will be covered in some detail include Hadoop (HDFS and Mapreduce), Hive, HBase, Pig, Mahout, Scribe , Zookeeper and Oozie/Azkhaban.

Requirements

Should have basic familiarity with Hadoop.

Speaker bio

Vinayak Hegde is Head of Engineering (Marketplace Management) at Inmobi. He has been active in opensource software community for more than a decade. He has been writing code in mainstream as well as esoteric programming languages on a variety of operating systems. He is a computer networking and data geek.

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hosted by

All about data science and machine learning