The Fifth Elephant 2016

India's most renowned data science conference

Let your Big Data Processing take flight with Apache Falcon

Submitted by Pallavi Rao (@pallavi-rao) on Thursday, 25 February 2016

videocam
Preview video

Technical level

Beginner

Section

Crisp talk

Status

Confirmed & Scheduled

View proposal in schedule

Vote on this proposal

Login to vote

Total votes:  +20

Abstract

At InMobi, a mobile advertising company, we see events arriving in excess of 10 billion per day. Analysis, reporting and inferencing from these requests (and responses served) is key to serving the right ad, to the right person, at the right time. We have nearly 200 complex big data pipelines that run against various data sources. Managing so many pipelines and the associated data was becoming a challenge and that is when we created an orchestration and data lifecycle management framework called Falcon. After benefiting immensely from this, we open sourced the product and it is now a Top Level Apache Project. The product was initially developed at InMobi Labs in India and now has many contributors and users from across the world. In this presentation we walk you through Apache Falcon and show how it has simplified our lives at InMobi.

Outline

The talk will mainly focus on the following areas:
1. Why did InMobi create Falcon and what are the features it offers
2. Overview of Falcon Architecture
3. How Apache Falcon has solved some big data processing problems at InMobi

Draft slides on slideshare

Speaker bio

Pallavi is an Architect at InMobi. She has been working on big data technologies for nearly 4 years now. She has deep knowledge of the Hadoop ecosystem, especially, YARN, PIG, Oozie, HBase, Hive and Storm. She is the committer for Apache Falcon and actively contributes to Apache PIG. She has spoken at conferences such as Annual RFID Conference , Information Management Technical Conference, ApacheCon (Big Data) and Grace Hopper Conference.

Slides

https://www.slideshare.net/secret/FxSYwsWYxLOzXb

Preview video

https://youtu.be/36DHGs9F9Oc

Comments

Login with Twitter or Google to leave a comment