Twitter today announced a new collaboration with Google that will see it moving a portion of infrastructure to Google’s Cloud Platform. The move is another high-profile win for Google in the cloud computing market, following its recent deal with Fitbit. Specifically, Twitter says it’s moving its cold data storage and its flexible compute Hadoop clusters – something it believes it needs to do in order to keep scaling its business.
Currently, Twitter’s Hadoop clusters are housed in its own physical data centers, the company told TechCrunch. This is a relatively small percentage of its total infrastructure, however – the bulk of its infrastructure will still remain in Twitter’s own data centers, we understand.
Twitter declined to share the impact of the move, in terms of cost savings or financial impacts.
The company publicly announced the news in a blog post this afternoon, where it noted it’s been assessing its platform and infrastructure needs over the past few years. It said it needs to be well-positioned to keep up with its growth.
Even though Twitter’s user growth has been flat, it sees hundreds of millions of tweets sent every day.
“The Hadoop compute system is the core of our data platform, and Twitter runs multiple large Hadoop clusters that are among the biggest in the world,” explained Twitter CTO Parag Agrawal, in the announcement. “In fact, our Hadoop file systems host more than 300PB of data across tens of thousands of servers,” he said.
Agrawal said that when the migration is complete, it will enable “faster capacity provisioning; increased flexibility; access to a broader ecosystem of tools and services; improvements to security; and enhanced disaster recovery capabilities.”
“Architecturally, we will also be able to separate compute and storage for this class of Hadoop workloads, which has a number of long-term scaling and operational benefits,” he added.
Twitter isn’t saying how long the migration will take.
The move to Google Cloud comes at a time when there’s heavy competition among cloud service providers – and Amazon is leading in terms of revenue. Grabbing a portion of Twitter’s business, then, is another big name win for Google, following its collaboration with Fitbit, announced just days ago.
“There is strong alignment with Twitter’s engineering strategy to meet the demands of its platform and the services Google Cloud offers at a global scale,” said Brian Stevens, Google Cloud CTO, in a statement. “Google Cloud Platform’s data solutions and trusted infrastructure will provide Twitter with the technical flexibility and consistency that its platform requires, and we look forward to an ongoing technical collaboration with their team.”
