Being a Product Engineer who uses serverless functions as a part of products I build, I closely experience how one can leverage serverless architectures to design a resource efficient and highly scalable infrastructure. The infrastructure provider we use which is AWS, provides a range of FaaS components from the popular lambda functions to other managed services like athena, kinesis firehose and quick sight. In this talk, as I give an introduction about serverless, We will walk through how we use them in production enabling resource optimization and low maintenance time. By the end of this talk we would’ve implemented an end to end logging pipeline and brought the generated sample logs to presentation tier for business insights. Not to mention, the system we setup can scale to handle 1000s of microservices and billions of log messages.

Outline:

Introduction
How our microservices looks like?
Architecting a logging framework
How the framework should be?
Metrics we needed
The conventional approach
Kafka, Cold storage, ELK
Problems we faced
Going Serverless
What is serverless
Advantages
The FaaS logging architecture
AWS Athena
AWS lambda
Kinesis firehose
Kinesis analytics
AWS quicksight
AWS s3
Demo
Caveats
Summary

Speaker bio

Naren is a Product Engineer with specific focus on building robust backend and scalable systems. He works on open source projects in his spare time. He loves speaking at tech conferences and currently helping MadStreetDen in scaling their Artifical Intelligence products. In his 4 years of industry experience he’s worn plenty of hats- like the one of a Trainer, Embedded Engineer and Backend/Product Engineer and sometimes even helmets- when he’s out cycling.
When he’s not stirring up code, you can find him whipping up a delicious gluten-free treat or travelling/cycling.

Miniconf on Cloud Server Management (Chennai)