The Fifth Elephant 2013

An Event on Big Data and Cloud Computing

Low Latency Access of Bigdata using Spark and Shark.

Submitted by Pradeep Kumar G.S. (@pradeep2002gs) on Monday, 29 April 2013

videocam_off

Technical level

Beginner

Section

Storage and Databases

Status

Submitted

Vote on this proposal

Login to vote

Total votes:  +23

Objective

This session aims at introducing latest Big data technology which involves in low latency access and in-memory data store using Spark framework.

Description

In this session we will have following sections: 1)Spark framework -1.a) Resilient Distributed Datasets - 1.b)Apache Mesos - Introduction 2)Mapreduce Vs Spark 3)Hive Vs Shark 4)Use Case : Data collection to Presentation using these framework. 5)Shark Demo:(if time permits)

Requirements

Basic understanding of hadoop and related big data technologies.

Speaker bio

This is Pradeep Kumar G.S working as Principal Database Engineer at Visual IQ ,Leader in Providing Digital Marketing Attribution ,I am Big data enthusiastic for past 3 years working on Bigdata Technology ,Conducted POC on hadoop and related ECO-system for VIQ. My final year M.S. project(Open Source Data warehousing as a Service for Enterprise) based on Spark,Shark, Open stack etc. Preparing to pursue PhD on Similar Area.

Links

Slides

http://www.slideshare.net/pradeep2002gs/low-latency-access-of-bigdata-using-spark-and-shark-20244610

Comments

  • 1
    t3rmin4t0r (@t3rmin4t0r) 5 years ago

    Are you planning to cover co-existing Spark on a hadoop cluster or is the plan to run a a mesos cluster instead?

    Spark is the only way (well, MPI is painful) for iterative algorithms which cannot be mapped to M-R, which is where the real value of the platform lies in. It would be good to cover the really unique aspects of Spark, as well.

  • 1
    Pradeep Kumar G.S. (@pradeep2002gs) Proposer 5 years ago

    Hi Gopal Vijayraghavan,

    Thanks for the comment. I have plan to cover architecture and functions of Spark along with Apache Mesos and Introduction about Shark(spark on hive), I want to show a case where hadoop and spark run on apache mesos side by side for business use case.
    Unique features of spark will be covered on Architecture section.

    Thanks, Pradeep Kumar G.S

Login with Twitter or Google to leave a comment