The Fifth Elephant 2016

India's most renowned data science conference

Satish Duggana


A large scale IOT platform architecture using open source apache projects like Nifi, Kafka, Storm, Spark and Hadoop.

Submitted Apr 30, 2016

Gartner predicts there will be 26 billion devices on the Internet of Things by 2020. Capturing and analyzing data from connected devices provides a wealth of opportunity. In this session we will look at how open source Apache projects like Apache NiFi, Kafka, Storm, Spark and Hadoop can work in concert to analyze and provide insights in a large scale distributed IoT architecture.


  • IoT Platform high level architecture
  • Edge processing (Apache NiFi)
  • Streaming and analytics (Kafka, Storm, Spark)
    • Ingestion
    • Transformation
    • Analytics
  • Notifications (Storm)
  • Alerts (Hadoop, Storm)
  • Storage (Hadoop)
  • Application development using SDK and UI

Speaker bio

Currently, helping design and development of IoT platform at Hortonworks and contributing various features in Apache Storm. Built large scale distributed systems, application servers, ESBs and backend infrastructure platforms for last 14 years in Hortonworks, BEA Systems/Oracle and Pramati Technologies.


{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}