Infrastructure Engineering @Scale 2020

Community to share and cherish their infrastructure stories and challenges at scale.

A Mighty elephant's journey to the cloud - How Twitter made the elephant fly?

Submitted by Abhishek J (@abhishekjt) on Jul 27, 2020

Status: Confirmed & scheduled

Abstract

The mission we serve at Twitter is to give everyone the power to create and share ideas and information instantly without barriers. And with people tweeting more and more every day, the data produced by these tweets starts to add up that calls for the need for more storage and more compute power. To keep processing massive amounts of data 24/7 and after a careful evaluation, we kicked off an initiative to move the Hadoop ad-hoc and cold clusters to Google cloud services. This initiative, which we fondly call as Partly cloudy. This talk provides insights into the current Hadoop infrastructure @Twitter and its journey to cloud.

Outline

  • Twitter Infrastructure workloads
  • Hadoop @Twitter scale
  • Partly cloudy - Journey to cloud
  • Q&A

Requirements

NA

Speaker bio

Abhishek J - SRE @Twitter
Jeffrey Jacson - SRE @Twitter

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('You need to be a participant to comment.') }}

{{ formTitle }}
{{ gettext('Post a comment...') }}
{{ gettext('New comment') }}

{{ errorMsg }}