A Mighty elephant's journey to the cloud - How Twitter made the elephant fly?
The mission we serve at Twitter is to give everyone the power to create and share ideas and information instantly without barriers. And with people tweeting more and more every day, the data produced by these tweets starts to add up that calls for the need for more storage and more compute power. To keep processing massive amounts of data 24/7 and after a careful evaluation, we kicked off an initiative to move the Hadoop ad-hoc and cold clusters to Google cloud services. This initiative, which we fondly call as Partly cloudy. This talk provides insights into the current Hadoop infrastructure @Twitter and its journey to cloud.
- Twitter Infrastructure workloads
- Hadoop @Twitter scale
- Partly cloudy - Journey to cloud
Abhishek J - SRE @Twitter
Jeffrey Jacson - SRE @Twitter