The Fifth Elephant 2012

Finding the elephant in the data.

What are your users doing on your website or in your store? How do you turn the piles of data your organization generates into actionable information? Where do you get complementary data to make yours more comprehensive? What tech, and what techniques?

The Fifth Elephant is a two day conference on big data.

Early Geek tickets are available from fifthelephant.doattend.com.

The proposal funnel below will enable you to submit a session and vote on proposed sessions. It is a good practice introduce yourself and share details about your work as well as the subject of your talk while proposing a session.

Each community member can vote for or against a talk. A vote from each member of the Editorial Panel is equivalent to two community votes. Both types of votes will be considered for final speaker selection.

It’s useful to keep a few guidelines in mind while submitting proposals:

  1. Describe how to use something that is available under a liberal open source license. Participants can use this without having to pay you anything.

  2. Tell a story of how you did something. If it involves commercial tools, please explain why they made sense.

  3. Buy a slot to pitch whatever commercial tool you are backing.

Speakers will get a free ticket to both days of the event. Proposers whose talks are not on the final schedule will be able to purchase tickets at the Early Geek price of Rs. 1800.

Hosted by

The Fifth Elephant - known as one of the best data science and Machine Learning conference in Asia - has transitioned into a year-round forum for conversations about data and ML engineering; data science in production; data security and privacy practices. more

Prashant Kumar

Building the Infrastructure to Handle Big Data

Submitted Jun 28, 2012

Understanding the pieces involved in building a generic Big data infrastructure

Outline

Description :
I’ll be sharing our experience at PromptCloud where we deal with terabytes of data daily. This talk will cover- how to build a multi-purpose technology stack to manage large scale data and the numerous processes that operate on it. We’ll cover following topics in various depths depending upon audience inclination:

  1. Automated installs for clusters - (Vagrant, Chef)
  2. Job scheduling (Resque/Celery)
  3. Process and cluster monitoring( God,Monit, Sensu), peripheral mention of Nagios
  4. Queue - the perennial suspect( RabbitMQ)
  5. Data storage - HBase, MongoDB, Voldemort, Riak
  6. Dashboard - Sensu

If there is time we can take up code integration and review (git,gerrit,jenkins) which are not directly needed but are crucial if the team size dealing with the above mentioned infrastructure increases.

Speaker bio

Speaker Bio :
Prashant passed out from IIT Kanpur and then joined Yahoo! data team where he worked for about a year. Among other things he worked on the analytics for the new version of Yahoo! front page then.

Currently Prashant runs a leading Big data and Cloud computing startup PromptCloud (http://promptcloud.com) where flagship product is large scale data crawl and extraction and hosted indexing of the data. PromptCloud also deals with Big data analytics but at a limited scale as of now. This talk will be based on many of the learnings at PromptCloud .

PS : I have submitted another talk for Riak, depending upon which (if any :) ) has enough interest , we’ll pick one.

Link for the Riak proposal - http://funnel.hasgeek.com/5el/394-riak-robust-and-featureful-distributed-key-value-store

Links

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hosted by

The Fifth Elephant - known as one of the best data science and Machine Learning conference in Asia - has transitioned into a year-round forum for conversations about data and ML engineering; data science in production; data security and privacy practices. more