PyConf Hyderabad 2017

Speed and agility are the most expected in today’s analytics tools. The quicker you get from idea to insights, the more you can innovate & perform ad-hoc data analysis. I will be talking about how we can use AWS serverless architecture to stream and analyze IoT data, managed by python. We can be up and running in minutes―starting small, but able to easily grow to millions of devices and billions of messages.

Outline

In this world of IoT, the hardware assembly of the tiny little sensors are just the tip of the iceberg. The real complexity lies in processing the massive amount of data these sensors generate. As I am typing this there are TeraBytes of data being generated every hour by connected things around the globe and the data is predicted to reach 403 ZetaBytes by 2018.

One cannot afford to spend most of the time on building and maintaining a backend servers to process this huge stream of data. The quicker you bring the data to the presentation tier, the more you can experiment and drive answers to new business questions. This is where we can leverage the agility of serverless architecture models. I will be talking about how AWS serverless architectures created and managed using python can help our case by taking care of the non-differentiated heavy lifting tasks such as managing servers, clusters, and device endpoints – allowing us to focus on assembling the IoT system, analyzing data, and building meaningful reports quickly and efficiently.

Outline :

1. The “thing” in “Internet of Things”
   a. Quick introduction to IoT. How much and what data they will generate
2. Let it in
   a. Designing a data pipeline to handle the large stream of data
3. Three-tier Architecture
   a. Introduction to one of the common n-tier architecture approach
4. Usual approach
   a. How we traditianally implement a 3-tier architecture in cloud to handle data
   b. Traditional architecture diagram (IoT gateway, kafka, storm, mongodb, cold storage)
   c. Drawbacks in this approach
5. Going serverless
   a. Advantages
   b. Using AWS serverless for our 3-tier architecture
6. Components
   a. Brief explanation of all the services we use in AWS serverless
   b. Services include AWS IoT, Kinesis Firehose, AWS lambda, S3, Athena, AWS Quicksight
7. Up and running in minutes (demo)
   a. Set up the serverless components on live using python and Boto3
   b. Get quick insights from mock data or data from sensors (using already setup Arduino/Raspi)

Speaker bio

An optimist by nature, Naren shows up late to meetings at MadStreetDen. In his 4 years of industry experience he’s worn plenty of hats- like the one of a Trainer, Embedded Engineer and Backend/Product Engineer and sometimes even helmets- when he’s out cycling. He is currently on a mission to help startups to design and write usable software even before making it reusable.
When he’s not stirring up code, you can find him whipping up a delicious gluten-free treat or travelling/cycling. He likes being asked about his endless love for Python through the handle @DudeWhoCode

Links

• www.dudewho.codes/talks
• www.github.com/dudewhocode

Slides

http://www.dudewho.codes/assets/pdfs/iot_meets_serverless.pdf