Semiconductor stocks have suffered an across-the-board selling in recent months thanks to the rapid deterioration of several end-markets (PC, smartphones…) and new export controls imposed by the US government on chips and equipment shipped to China. But this “implosion” could also create buying opportunities as some end-markets and, more specifically, the data center one enjoys secular tailwinds.
Even if data center spending will not be immune to current macro headwinds, the emergence of accelerators is indeed expected to make Artificial Intelligence (AI) mainstream in coming years and, accordingly, to sustain demand growth for specific chips.
Besides easing our day-to-day lives (targeted advertisement, real-time natural language processing and translation, profile-matching on dating apps…), AI is also expected to help us overcome some global issues like efficient usage of water resources with smart irrigation systems, reduced waste by enhanced material allocation or energy savings through smart grids. Ultimately, AI will also contribute to design fusion reactors and master quantum computing which will instantly put humanity on a whole new technological level. However, this will require an ever-growing need for data storage, more efficient computing power, and lightspeed networking capabilities.
As their name implies, general purpose microprocessors (CPUs) are good all-around processing chips but are rapidly showing their weaknesses for very specific tasks. This is the main reason why engineers added coprocessors inside them to solve this inefficiency. An additional way of unloading “non-pure” computing tasks from CPUs is to couple them with so-called accelerators like, for example, GPUs for 3D graphics processing or DSPs (signal processing chips) for sound enhancement.
In the specific case of AI, the training of state-of-the-art Deep Learning (DL) models, which are now dealing with billions of parameters, also necessitates dedicated accelerators. For example, the management of network traffic and the storage of the huge amount of data needed to run these algorithms is now gradually outsourced to SmartNICs (see our recent article). More importantly, the calculation of the giant matrices (or tensors to be more precise), which are defining DL models’ knowledge, has now also been outsourced to customized components.
Initially, researchers trained their models with CPUs but gradually switched to GPUs as their superior parallel computation power better fits the type of calculations involved in AI. For more flexibility, fully programmable chips, or FPGAs, are also used by some players (mainly by Microsoft). Nowadays, highly specialized chips (ASICs), in which the structure of deep learning models have been directly “hard-coded” in their silicon, are clearly the way to go to stay at the leading edge of AI.
These AI accelerators, commonly called Tensor Processing Units (TPUs) or Neural Processor Units (NPUs), are bringing a very significant improvement in terms of speed and energy consumption during the training phase of AI models (which could last more than a full week for massive DL models, even on the world’s most powerful systems). The huge amounts of data – the necessary “fuel” to train deep learning models – accumulated by hyperscalers like Google, Facebook or Amazon helped them to be at the forefront of last decades’ AI developments. So, designing their own proprietary AI chips was just a natural extension for them. Google started the trend in 2016 with its own TPUs, followed by Baidu in 2018 (Kunlun chips) and, more recently, Amazon with its Trainium NPU.
Web giants are running their in-house developed NPUs exclusively inside their data centers to improve their own products/services (better search results, more accurate product recommendations…) and to rent out their top-notch AI computing platforms to companies, universities, and governments. AI cloud-based services like Google’s Vertex or AWS’ Machine Learning are expected to grow at a CAGR of 21% with a market of more than USD 10 billion in 2026 according to Technavio.
Obviously, legacy CPU and GPU companies like Intel, Nvidia or AMD didn’t stay idle and entered the AI chip race several years ago by either leveraging their existing technologies (e.g., Nvidia’s CUDA) or through acquisitions (Intel’s Habana). Many startups like Cerebras or Graphcore also entered the party of this booming semis segment expected to grow at more than 18% per annum to reach a size of $80 billion by 2027 according to IndustryARC.
Once AI models have been trained to deliver satisfactory results, they are implemented in real-life situations for which they were optimized. This is called inference, as the models infer the most appropriate course of action when presented with stimulus coming from human inputs, sensors, cameras… Practically, the definition (the giant matrices) of the trained AI models are uploaded into specific chips, sometimes called Edge TPUs/NPUs. These are powering the AI features of smartphones like face recognition or the assisted/autonomous flying skills of drones.
Here, on the inference side, engineers are designing these chips with the objectives of very low energy consumption, reduced latency, and small physical size. The edge TPU market is populated by the usual suspects like Intel (Greco chip), Amazon (Inferenzia TPU), Nvidia (which sticks to its GPU-based design), Qualcomm or Apple with its in-house developed Neural Engine. To the contrary of the very complex (both in terms of design and manufacturing) and hence pricey training TPUs, inference TPUs are cheap mass market chips. An armada of IoT devices are already or will extensively use Edge TPUs, another 20%+ annual growth driver for the semiconductor industry.
In conclusion, while we would expect data center spending to decelerate given softening macro trends, we believe it should keep enjoying robust growth thanks to secular tailwinds and, specifically, the increasing adoption of AI powered by accelerators. Companies exposed to this end-market such as AMD, Nvidia, Intel and Marvell should then avoid some of the massive EPS cuts that have plagued the industry recently.
Importantly, their valuations are back to very attractive levels (11.5x 2023 PE on average for AMD, Intel, and Marvell; Nvidia still remaining expensive at 30x), suggesting that most of the current risks are already well priced.