##About Rootconf Pune:
Rootconf Pune is a conference for:
- DevOps engineers
- Site Reliability Engineers (SRE)
- Security and DevSecOps professionals
- Software engineers
- Network engineers
The Pune edition will cover talks on:
- InfoSec and application security for DevOps programmers
- DNS and TLS 1.3
- SRE and distributed systems
- Containers and scaling
Speakers from Flipkart, Hotstar, Red Hat, Trusting Social, Appsecco, InfraCloud Technologies, among others, will share case studies from their experiences of building security, SRE and Devops in their organizations.
##Workshops:
Two workshops will be held before and after Rootconf Pune:
- Full-day Prometheus training workshop on 20 September, conducted by Goutham V, contributor to Prometheus and developer at Grafana Labs. Details about the workshop are available here: https://hasgeek.com/rootconf/2019-prometheus-training-pune/
- Full-day DNS deep dive workshop on 22 September by Ashwin Murali: https://hasgeek.com/rootconf/2019-dns-deep-dive-workshop-pune/
##Event venue:
Rootconf Pune will be held on 21 September at St. Laurn Hotel, Koregaon Park, Pune-411001.
#Sponsors:
Click here to view the Sponsorship Deck.
Email sales@hasgeek.com for bulk ticket purchases, and sponsoring the above Rootconf Series.
Rootconf Pune 2019 sponsors:
#Platinum Sponsor
 |
 |
|
#Bronze Sponsors
 |
 |
 |
#Community Partner
 |
 |
 |
##To know more about Rootconf, check out the following resources:
- hasgeek.com/rootconf
- hasgeek.com/rootconf/2019
- https://hasgeek.tv/rootconf/2019
For information about the event, tickets (bulk discounts automatically apply on 5+ and 10+ tickets) and speaking, call Rootconf on 7676332020 or write to info@hasgeek.com
AK
Arya Ketan
@aryaketan
Abstractions of a Managed Stream Processing platform and how we provide them at scale in Flipkart.
Submitted Apr 1, 2019
Section:
Full talk of 40 mins duration
Technical level:
Advanced
Session type:
Full talk of 40 mins
Section:
Full talk (40 mins)
Category:
Distributed systems
We live in an age of ML models, deeply personalised user experiences and quick data driven business decisions. The common denominator enabling all of it is data processing systems, especially real time ones.
We at Flipkart use streaming systems for a variety of real time computations like analytics and reporting in flash sale events, annual Big Billion day sales or personalisation of search and browse experience. These use-cases requires stateful stream processing (like - stream joins and time windowed aggregates) at a very high scale and such systems becomes very complex very fast.
Problem Statement:
While there are many stream processing engines in the open source / closed source community, they are not a platform and do not provide the abstractions that a stream platform requires. An Ideal stream processing platform requires
a) A good programming model
b) Stateful operations
c) Low Entry Bar
d) Infrastructure Management
e) Monitoring & Alerting
f) Job Lifecycle Management
Enter fStream :
Most of the stream processing engines do not cater to all these and focus on few of the capabilities.
This motivated us to build fStream, a managed stateful stream processing platform which aims to fill this gap.We built fStream to abstract out above complexities and provide a simple declarative interface to define powerful computation graphs (DAG) and execute it without worrying about the underlying setup, infrastructure and scale.
In this presentation, we will talk about a few e-commerce domain problems like contextual search, personalisation, analytics and reporting requirements at high scale ‘sale events’ and how we solve them through stateful processing system like fStream.
We will discuss the stream processing evolution from the days of Storm to now Flink/Beam and explain what aspects of the stream processing platform requirements they fulfil and which ones they lack. We will then talk about the architecture, interfaces and management layers of fStream which is aimed at simplifying the whole lifecycle of streaming jobs (creation, deployment, monitoring and maintenance).
Key take-aways for the audience would be
- Patterns and Paradigms of an Ideal stream processing platform.
- Why computing on Storm / Spark or Flink is not enough
- Architectural solutions that fStream , a managed stateful stream processing platform provides.
Outline
Agenda for the talk would be :
- Stream Processing use-cases and examples from Flipkart.
- Why a stream platform?
- FStream - Managed Stateful Stream Processing Platform at Flipkart.
- FStream Components.
Speaker bio
Arya Ketan has been part of Flipkart since its early days and is currently a software architect. He is passionate about developing features and debugging problems in large scale distributed systems. Nowadays, he is working in the big data platform of Flipkart which powers near real time and batch computation on eCommerce datasets. He completed his bachelors in engineering from NIT,Trichy,India in 2008.
{{ gettext('Login to leave a comment') }}
{{ gettext('Post a comment…') }}{{ errorMsg }}
{{ gettext('No comments posted yet') }}