Google is the third largest designer of processors for datacenters

Think of Google, and what comes to mind? Surely the image of the leading Internet search engine of recent times and, more recently, cloud computing and Artificial Intelligence (AI) will come to mind. Would it surprise you to know that Google has also been working for almost a decade on the design of AI accelerators for datacenters since it launched the Tensor Product Unit (TPU) in 2015, an application-specific integrated circuit (ASIC) designed specifically to accelerate machine learning (ML) workloads?

According to TechInsights, Google is currently the third largest designer of processors for datacenters, with a market share comparable to that of more prominent companies such as Intel or AMD. And it is expected to continue growing throughout 2024. Google’s use of customised chips in its own datacenters reached the 2 million mark last year, a figure that puts it behind only NVidia and Intel in terms of market share, according to an analysis by TechInsights.

According to TechInsights’ assessment, Google’s TPU generations have contributed to the company’s growth year after year. With TPU v4 introduced in 2021 and the emergence of major language models, Google’s processor business has grown significantly.

To further stimulate this movement, Google even hired an Intel veteran last March to head a new division that will develop customised system-on-chips (SoCs) for the company’s datacenters. It also plans to recruit hundreds of engineers to create new highly integrated circuits and replace Google’s server motherboards by the middle of the decade, according to an article on the Tom’s Hardware website.

Market trends

Providers of hyperscale cloud services, the so-called hyperscalers, such as Google, Meta, Microsoft and Amazon, are investing strategically in the development of proprietary semiconductor technologies in response to the growing demand for GPUs in the Generative AI (GenAI) market.

In addition to Google, Microsoft and AWS have also made inroads into the world of proprietary chips. Last November, during the Ignite conference, Microsoft unveiled the Azure Cobalt 100 CPU based on Arm technology for generic workloads and the Azure Maia 100 AI accelerator for use in the Azure cloud and optimised for Generative AI. In the same month, AWS presented the fourth generation of its home-grown processor, Graviton4, also based on Arm technology. According to the company, since its launch in 2018, more than two million Graviton chips have been deployed to more than 50,000 customers and 150 instance types.

According to GlobalData, this move towards customised chips aims to reduce dependence on NVidia and also to promote innovation and facilitate global expansion. In addition, it seeks to reduce the financial costs of acquiring more expensive processors to meet the growing demand for AI services faced by large cloud computing companies.

‘There is a significant imbalance between supply and demand when it comes to GPUs, because Generative AI models in general, and especially multimodal systems that produce images and videos, heavily exploit the parallel processing capabilities of GPUs. And these chips are expensive and scarce,’ explains Beatriz Valle, senior technology and business services analyst at GlobalData.

‘To address this trend, hyperscalers that provide AI services are adopting proprietary technologies to run AI workloads. Google has its TPUs; Amazon has its Inferentia and Trainium architectures. Meta recently announced a new generation of customised chips to help drive AI-based rankings and ads on its social media platforms,’ she recalls.

Partnership between AWS and NVidia

During the launch of the new Blackwell super GPU in March, NVidia declared that the new processor would be able to run LLM models with trillions of parameters and up to 25 times less cost and energy consumption than its predecessor. At the time, Jensen Huang, NVidia’s CEO, mentioned that many organisations were waiting to adopt the new Blackwell, including AWS, alongside Dell Technologies, Google, Meta, Microsoft, OpenAI, Oracle, Tesla and xAI.

‘For three decades we have pursued accelerated computing with the goal of enabling transformative breakthroughs like deep learning and Artificial Intelligence. Generative AI is the defining technology of our time, and Blackwell is the engine that will power this new industrial revolution. Working with the world’s most dynamic companies, we will fulfil the promise of AI across all industries,’ said Huang. ‘The new Blackwell GPU will work very well on AWS. And that’s why NVidia chose AWS to co-develop Project Ceiba, combining Blackwell superchips with AWS Nitro System’s advanced virtualisation and Elastic Fabric Adapter’s ultra-fast networking for NVidia’s own AI research and development. Through this joint effort between AWS and NVidia engineers, we will continue to innovate together to make AWS the best place for anyone to use NVidia GPUs in the cloud,’ said Andy Jassy, president and CEO of Amazon.