The Who’s Who of High-Performance Computing (HPC) is currently gathering in Colorado, US for the annual SuperComputing conference (SC23).
Halfway through the conference, one trend is pretty clear: the whole IT industry is all in on Generative AI, starting from the chip designers up to the large system builders.
Nvidia teased its upcoming server-class GPU architecture called Blackwell but also showcased the next iteration of its successful H100 chip (simply called H200) which will be the first to offer the latest generation of high-bandwidth memory (HBM). Partner SK Hynix will provide the HBM3e memory modules, helping Nvidia to double the GPU’s bandwidth over the previous generation.
As we have already mentioned, the mature and hence “boring” memory segment is literally getting a steroid shoot from the explosive storage needs of Large Language Models (LLM) à la ChatGPT.
In 2024, the AI computing market will not be a one-horse market anymore as Intel and AMD will respectively release the Gaudi3 (version 2 is already very competitive as demonstrated by the last MLPerf benchmark) and the long-awaited Instinct MI300 AI accelerators. Rapid AI developments, but also Increased competition explain why Nvidia has recently decided to accelerate its product refresh frequency from a biennial to an annual rhythm.
The server-class CPU market is also gaining traction with Intel releasing its 5th generation of Xeon chips, which will finally be competitive against AMD’s offering (in terms of computing power and energy consumption) and hyperscalers’ in-house developments like Amazon’s Graviton. Furthermore, as these new chips will integrate AI-specialized instructions, they will be in a favorable position to serve the very large inference market (both on the client and the server sides).
All of these energy-hungry chips (the H100 can suck more than 650 Watts…) cannot be cooled anymore with air flows but require liquid cooling solutions. The Generative AI triggered race towards increasingly powerful AI super calculators is hence boosting this specific segment with every system builder from MSI, Asus, Gigabyte to Tyan releasing direct liquid cooling and/or immersion cooling products.
According to a just published research report by Global Market Estimates, the global market for AI-optimized hardware (which includes GPUs, TPUs, high-speed memory, parallel processing capabilities, energy efficiency, and customized architecture) is expected to grow at an annual rate of 25%+ over the next 5 years.
Interestingly, while much of the focus until now has been on Nvidia, this growth should now benefit a larger number of semiconductor companies and put the spotlight on rival chips (AMD, Intel) and other technologies (memory, cooling…).
Generative AI’s revenue generating potential and/or cost cutting capabilities have a clear disruptive effect on the IT industry as well as many other economic segments. We believe that the impact will be as profound as mobile communications or the Internet and is already reshaping the HPC and cloud computing infrastructures.
