Miniconf on Cloud Server Management (Chennai)

Serverless architectures has been around for past few years and there has been quite a few skepticism surrounding it. Few might argue that it’s just another buzzword for marketing. But serverless architectures offer more than a catchy buzzword. In this talk we will discuss, what is serverless, when to and when not to use them and how can we use Amazon Web Services to implement a real-time, production grade serverless logging pipeline. By the end of the talk, audience will get an introduction to serverless and also get to know how to design, deploy and scale infrastructures using the same.

Outline

Being a Product Engineer who uses serverless functions as a part of products I build, I closely experience how one can leverage serverless architectures to design a resource efficient and highly scalable infrastructure. The infrastructure provider we use which is AWS, provides a range of FaaS components from the popular lambda functions to other managed services like athena, kinesis firehose and quick sight. In this talk, as I give an introduction about serverless, We will walk through how we use them in production enabling resource optimization and low maintenance time. By the end of this talk we would’ve implemented an end to end logging pipeline and brought the generated sample logs to presentation tier for business insights. Not to mention, the system we setup can scale to handle 1000s of microservices and billions of log messages.

Outline:

1. Introduction
2. How our microservices looks like?
3. Architecting a logging framework
4. How the framework should be?
5. Metrics we needed
6. The conventional approach
7. Kafka, Cold storage, ELK
8. Problems we faced
9. Going Serverless
10. What is serverless
11. Advantages
12. The FaaS logging architecture
13. AWS Athena
14. AWS lambda
15. Kinesis firehose
16. Kinesis analytics
17. AWS quicksight
18. AWS s3
19. Demo
20. Caveats
21. Summary

Speaker bio

Naren is a Product Engineer with specific focus on building robust backend and scalable systems. He works on open source projects in his spare time. He loves speaking at tech conferences and currently helping MadStreetDen in scaling their Artifical Intelligence products. In his 4 years of industry experience he’s worn plenty of hats- like the one of a Trainer, Embedded Engineer and Backend/Product Engineer and sometimes even helmets- when he’s out cycling.
When he’s not stirring up code, you can find him whipping up a delicious gluten-free treat or travelling/cycling.

Links

• Website: http://www.dudewho.codes
• Previous talks: http://blog.dudewho.codes/talks
• Github: https://www.github.com/DudeWhoCode

Slides

https://docs.google.com/presentation/d/1B7NhNDoSqwXXBDQvYoWIFrwE6Cd2_w7wdYARG-i-g9M/edit?usp=sharing