Generative AI: do data centres need to be revamped?

During the release of Nvidia’s latest financial report, Jensen Huang, the company’s founder and CEO, made a statement that has left the datacenter world on alert. The executive said that the “computer industry is going through two simultaneous transitions – accelerated computing and generative AI”. He went on to say that “a trillion dollars of the world’s installed datacenter infrastructure will transition from the general purpose model to accelerated computing as companies race to apply generative AI across their products, services, and business processes”.

Will datacenters really need to be completely overhauled because of the widespread use of generative Artificial Intelligence (AI) models? It is certain that the more companies integrate AI tools into their business routine, the greater the demand for datacenters resources will be. And what about existing infrastructures? Will they be able to keep up? And at what cost?

According to a forecast by Tirias Research, the costs of data center server infrastructure for generative AI added to operational expenses will exceed $76 billion by 2028. This increase in costs could end the dream of companies that, for example, seek to automate their business by incorporating this type of AI tool. According to this Tirias perspective, the incorporation of a new datacenter infrastructure would offer a four-fold increase in computational performance, but this gain would be outweighed by a 50-fold increase in AI workloads, even with innovative and more efficient inference algorithms.

In an interview with Tech Wire Asia, Chris Street, director of data centres at real estate investment consultancy JLL, commented that the industry is experiencing an increase in power density aimed at supporting AI applications and this brings with it challenges, especially older facilities not designed for such use. “This situation makes the move to cloud services imperative for many organisations,” says the analyst. For Street, some facilities can be reused, while others may require considerable effort to upgrade and may not be worth it.

Bradley Shimmin, meanwhile, an analyst in the data and AI sector at advisory group Omdia, acknowledges that there may be a need to adopt new approaches to acceleration hardware, however he does not fully share Nvidia’s Huang’s view that data centres will need to replace all their equipment.

On the Data Center Knowledge website, Shimmin detailed that for many use cases, especially those using highly computationally demanding training models, companies will need to invest in the best types of hardware acceleration for AI. “However, there is an opposite trend happening right now, in which researchers are learning to do more with fewer models and with fewer parameters, highly selected datasets and smarter training, for example,” Shimmin added.

Another ecosystem initiative that can help datacenters support generative AI use cases comes from processor manufacturers, the Omina executive highlights. For example, Samsung is striving to run AI models at the chip and edge device level, thus easing the burden on datacenters.

However, there are those who agree with the Nvidia executive. Karl Freund, founder and principal analyst at Cambrian-AI Research, told Data Center Knowledge that “Huang is a visionary and has long said that data centers must be accelerated.” At least for Nvidia, that scenario is coming true, a beautiful reality that is demanding a lot of specialized AI hardware for datacenters – in its latest financial report, the company reported record revenue of $4.28 billion from its data center division, up 14% year-over-year and 18% quarter-over-quarter.

Why is Nvidia riding the AI wave?

Nvidia’s hardware acceleration solutions are riding the AI wave so well as to cause the company to reach a market value of $1 trillion today, reaching what they call the “12-zero club”.

But how did a company founded in 1993 by a Taiwanese-American electrical engineer named Jensen Huang as a focus on high-resolution graphics processing reach the world of Artificial Intelligence and datacenters?

Well, graphics processing units (GPUs) are designed to perform endless mathematical calculations using large volumes of data (pixels in images) as input to render three-dimensional spaces in an optimized and accelerated manner, unlike CPUs which are meant for generic use. Over time, GPUs, which were and are until today in the core business of Nvidia, evolved and began to have several processors running smaller calculations in parallel, derived from a larger and more complicated problem, such as images with great detail in accelerated motion under player interaction, for example.

We can now say that the game has turned somehow. To the NetworkWorld website, Manuvir Das, Nvidia’s vice president of corporate computing, said the company “is definitely focused on the corparative world”, even though Nvidia’s essence of being a gaming organisation won’t change.

Returning to the current AI landscape, coincidentally, machine learning and its training algorithms, present in LLM models, also require many simultaneous computations using large volumes of data. It’s not hard to deduce that Nvidia’s GPUS would be well suited for use with AI. “They were smart, capitalising on that,” says Willy Shih, a professor at Harvard Business School.

According to Das, Nvidia’s suite of GPUs offer different functionalities depending on the target market. Enterprise GPUs have an engine for natural language processing, for example, and other functions not found in gaming GPUs.

Nvidia has come out ahead in this race, but Intel, AMD, as well as Amazon and Google, are also investing in the segment of GPUs for AI.