Long Running Services on YARN: Future of Service Deployment & Management via Hadoop

Jul 2016

25 Mon

26 Tue

27 Wed

28 Thu 08:30 AM – 06:25 PM IST

29 Fri 08:30 AM – 06:15 PM IST

30 Sat 08:45 AM – 05:00 PM IST

31 Sun 08:15 AM – 06:00 PM IST

Make a submission

NIMHANS Convention Centre

The Fifth Elephant is India’s most renowned data science conference. It is a space for discussing some of the most cutting edge developments in the fields of machine learning, data science and technology that powers data collection and analysis.

Machine Learning, Distributed and Parallel Computing, and High-performance Computing continue to be the themes for this year’s edition of Fifth Elephant.

We are now accepting submissions for our next edition which will take place in Bangalore 28-29 July 2016.

#Tracks

We are looking for application level and tool-centric talks and tutorials on the following topics:

Deep Learning
Text Mining
Computer Vision
Social Network Analysis
Large-scale Machine Learning (ML)
Internet of Things (IoT)
Computational Biology
ML in healthcare
ML in education
ML in energy and ecology
ML in agriculrure
Analytics for emerging markets
ML in e-governance
ML in smart cities
ML in defense

The deadline for submitting proposals is 30th April 2016

Format

This year’s edition spans two days of hands-on workshops and conference. We are inviting proposals for:

Full-length 40 minute talks.
Crisp 15-minute talks.
Sponsored sessions, 15 minute duration (limited slots available; subject to editorial scrutiny and approval).
Hands-on Workshop sessions, 3 and 6 hour duration.

Selection process

Proposals will be filtered and shortlisted by an Editorial Panel. We urge you to add links to videos / slide decks when submitting proposals. This will help us understand your past speaking experience. Blurbs or blog posts covering the relevance of a particular problem statement and how it is tackled will help the Editorial Panel better judge your proposals.

We expect you to submit an outline of your proposed talk – either in the form of a mind map or a text document or draft slides within two weeks of submitting your proposal.

We will notify you about the status of your proposal within three weeks of submission.

Selected speakers must participate in one-two rounds of rehearsals before the conference. This is mandatory and helps you to prepare well for the conference.

There is only one speaker per session. Entry is free for selected speakers. As our budget is limited, we will prefer speakers from locations closer home, but will do our best to cover for anyone exceptional. HasGeek will provide a grant to cover part of your travel and accommodation in Bangalore. Grants are limited and made available to speakers delivering full sessions (40 minutes or longer).

Commitment to open source

HasGeek believes in open source as the binding force of our community. If you are describing a codebase for developers to work with, we’d like it to be available under a permissive open source licence. If your software is commercially licensed or available under a combination of commercial and restrictive open source licences (such as the various forms of the GPL), please consider picking up a sponsorship. We recognise that there are valid reasons for commercial licensing, but ask that you support us in return for giving you an audience. Your session will be marked on the schedule as a sponsored session.

Key dates and deadlines

Revised paper submission deadline: 17 June 2016
Confirmed talks announcement (in batches): 13 June 2016
Schedule announcement: 30 June 2016
Conference dates: 28-29 July 2016

##Venue
The Fifth Elephant will be held at the NIMHANS Convention Centre, Dairy Circle, Bangalore.

##Contact
For more information about speaking proposals, tickets and sponsorships, contact info@hasgeek.com or call +91-7676332020.

Hosted by

The Fifth Elephant

The Fifth Elephant - known as one of the best data science and Machine Learning conference in Asia - has transitioned into a year-round forum for conversations about data and ML engineering; data science in production; data security and privacy practices. more

All submissions

Previous Next

Long Running Services on YARN: Future of Service Deployment & Management via Hadoop

Submitted Mar 14, 2016

Technical level: Advanced

YARN has long aspired to be an operating system for the data center. In order to bring that promise to fruition, it must be able to host services that transcend the usual provision-execute-teardown lifecycle of most Hadoop processing frameworks. In this talk, we will share what we’ve learned, building long running services together with on-demand scaling and monitoring on YARN. We will first discuss using Slider for packaging and deploying persistent services such as HBase and memcached. Next, we will explore using the YARN mechanisms for log aggregation. We will then detail our experiences running ZooKeeper as a long running, dynamically reconfigurable service in YARN (ZOYA). Finally, we will focus on how best to set up monitoring for these types of services with emphasis on JMX based appenders and metrics via Ganglia.

As this is a nascent field, our work in this area has exposed us to a number of potential areas of improvement in YARN and Hadoop. We will conclude with a summary of our findings and pointers to related work.

Outline

How to productionize long running services on YARN - Services like HBase, Cassandra, SOLR etc.

Requirements

Basic understanding of Hadoop and Yarn

Speaker bio

Prasath Venkatraman is the co-founder of Zudly Labs. Over the last 8 years, he has worked on various distributed data systems and full stack application development. Prasath is also the Founding Engineer at Milaap.org (India’s leading Micro lending platform) and played various key engineering roles at Akamai & Flipkart. More recently, as a consultant for Altiscale R & D he has been working on various Slider applications with an emphasis on HBase. At Zudly Labs, he is currently focused on building a real-time pipeline for ingesting events data into Hadoop for batch and stream processing.