The Fifth Elephant 2012

Finding the elephant in the data.

A Herd of Elephants - Navigating the Hadoop Ecosystem

Submitted by Vinayak Hegde (@vin) on Wednesday, 25 April 2012

videocam_off

Technical level

Intermediate

Section

Big Data Infrastructure & Processing

Session type

Tutorial

Status

Confirmed

Vote on this proposal

Login to vote

Total votes:  +23

Objective

A overview of the Hadoop ecosystem and how the different parts of the ecosystem interact and fit together.

Description

Hadoop has matured to point where it is not longer just one project but a bunch of projects ranging from getting data onto the cluster to processing and analyzing data to managing the cluster itself. I will be talking from my personal experiences from setting up a hadoop cluster at Inmobi that processes 10TB+ of Data per day (and growing). The several Hadoop clusters in Inmobi are spread over multiple datacenters across continents.

Projects that will be covered in some detail include Hadoop (HDFS and Mapreduce), Hive, HBase, Pig, Mahout, Scribe , Zookeeper and Oozie/Azkhaban.

Requirements

Should have basic familiarity with Hadoop.

Speaker bio

Vinayak Hegde is Head of Engineering (Marketplace Management) at Inmobi. He has been active in opensource software community for more than a decade. He has been writing code in mainstream as well as esoteric programming languages on a variety of operating systems. He is a computer networking and data geek.

Comments

  • 1
    Vinayak Hegde (@vin) Proposer 6 years ago

    Slides have been posted to git hub : https://github.com/vinayakh/talks/blob/master/the5el/herd-of-elephants/Apache%20Hadoop%20-%20A%20Herd%20of%20Elephants.pdf

Login with Twitter or Google to leave a comment