From Application Developer to Big Data Engineer
Submitted by Chandulal Kavar (@chandukavar) on Tuesday, 13 March 2018
In this talk, I will talk about my journey from working as an App Dev to a Data Engineer at ThoughtWorks. This transition helped me to change my mindset around Big Data as well as problem-solving techniques required for distributed computing problems. I will also cover how one API over other will affect the performance and memory usage. Furthermore, I will show some of these techniques through some examples.
- What people do when they start exploring Big Data?
- Why people quit?
- Mindset Change
- Silo Computing vs Distributed Computing
- Log vs Logging
- Data quality in Big Data
- Problem Solving Techniques for Distributed computing problems
- Choosing one API over Other
- How to introduce new Big Data tool or framework
I am a Data Engineer at Thoughtworks, designing and building data-intensive and scalable Data Platforms. I love to solve the problems that run on Big Data processing frameworks. I am a blogger too, you can find my blogs on medium @chandukavar