48 million chips: A 20,000-word analysis of data centers

Recently, the Semiconductor Industry Association (SIA) partnered with Deloitte to release a new report, which notes that all types of semiconductor technologies are an integral part of artificial intelligence (AI), account for a significant share of the value of AI infrastructure, and will bring enormous market opportunities in the coming years.

This new report points out that chips account for over 95% of the component value in leading AI server racks, and more than 50% of the total capital expenditure required to build and operate an AI data center. The study also details how AI requires a full range of semiconductor technologies including logic, memory, analog, and base chips. Furthermore, the report forecasts that the annual revenue of semiconductors used in AI data centers could reach $1.2 trillion by 2028. This represents an almost tenfold growth over the past four years, and is more than 50% higher than the total global semiconductor sales across all end-use applications in 2025.

Other Key Findings from the Report:

1. AI data centers require massive amounts of computing, storage and memory bandwidth, power distribution, and network capabilities, all of which are enabled by a full stack of chip technologies;

2. A complete AI server rack contains more than 4,500 packaged chips in total. These include:

(1) Advanced logic chips, such as AI accelerators, application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), central processing units (CPU), data processing units (DPU), and network chips.

(2) Memory, such as high-bandwidth memory (HBM), dynamic and static random access memory (DRAM and SRAM), and non-volatile flash memory (NAND).

(3) Analog and base chips, such as power management chips, transceivers, controllers, and sensors.

The report also analyzes future market trends in the AI data center sector, with a particular focus on the rapid transformations of AI in the coming years that will bring both challenges and opportunities to the semiconductor industry. As AI continues to grow rapidly, competition for global leadership will intensify, and technological transformation will require sustained innovation.

Compiled Full Text of the Report:

Abstract

Semiconductors are the foundation of artificial intelligence (AI), a technology that is transforming our economy and society, making entire industries more productive and innovative, and driving major scientific breakthroughs.

Today's AI systems are built on decades of innovation across the entire semiconductor ecosystem. As chip technology continues to advance, AI will become more powerful, energy-efficient, and cost-effective. In turn, more powerful AI will help improve chip design, optimize semiconductor manufacturing, and drive greater demand for a wide range of AI chips.

Key Takeaways:

1. Semiconductors are the fundamental enabling technology for AI. Chips form the foundational hardware layer of modern AI systems and account for a significant share of the total value of modern AI servers:

A single AI server rack contains more than 4,500 packaged chips, made up of approximately 20,000 individual die — unique integrated circuits.

Semiconductors account for more than 95% of the bill of materials value in leading AI server racks, and more than 50% of the total capital expenditure required to build and operate an AI data center.

2. AI requires a full suite of semiconductor technologies.

To run complex AI training and inference workloads, today's AI data centers require massive amounts of computing, storage and memory bandwidth, power distribution, and networking capabilities — all of which are provided by a full stack of chip technologies. Every type of chip technology is critical to advancing AI development, and critical dependency issues in any one area could slow this development process. The chips in an AI data center include:

Advanced logic chips, such as AI accelerators, application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), central processing units (CPU), data processing units (DPU), and network chips.

Memory, such as high-bandwidth memory (HBM), dynamic and static random access memory (DRAM and SRAM), and non-volatile flash memory (NAND).

Analog and base chips, such as power management chips, transceivers, controllers, and sensors.

3. AI is the primary driver of chip demand across the entire semiconductor industry.

In a positive feedback loop, advances in AI drive demand for higher-performance and more efficient semiconductors, while advances in semiconductor technology enable more powerful and advanced AI systems.

To meet global demand for new AI applications, governments and industry will invest more than $4 trillion in new data center infrastructure by 2028, with up to $2.8 trillion of that investment allocated to semiconductors.

By 2028, annual revenue from semiconductors deployed in AI data centers could exceed $1.2 trillion, an almost tenfold growth in four years.

The AI data center market is experiencing unprecedented growth, with a projected compound annual growth rate (CAGR) of 88.8% from 2022 to 2028. While initial growth was driven by the rapid adoption of generative AI, sustained demand remains strong, with a projected CAGR of 56.3% from 2025 to 2028.

The entire semiconductor supply chain underpins the construction of AI infrastructure. No semiconductors, no AI. To lead this transformative technology, governments and industry must work together to advance policies that accelerate the comprehensive development and innovation of chip technology, and collaborate closely with global partners to build a strong and resilient supply chain.

The Many Chips at the Core of AI

In recent years, artificial intelligence (AI) has experienced explosive growth, and the developers training and deploying AI models have received extensive attention. A wide range of semiconductors are the backbone and enabling technology of the AI hardware stack, and advances in chip technology have driven improvements in processing power, computing efficiency, and overall performance of AI applications. Semiconductors form the foundation of the AI systems embedded in our everyday digital experiences.

This report provides a unique, in-depth analysis of the various chips that form the core of AI infrastructure by virtually tearing down a state-of-the-art AI data server, the fundamental building block of modern AI infrastructure. Unlike traditional analyses that only focus on system-level performance or market size, this report dives deep into the semiconductor components inside each subsystem of the server, mapping out the chips, die, and supporting components that power today's data centers.

We further complement this analysis by focusing on where value is concentrated in these server systems, and which technologies are critical, ranging from cutting-edge logic built on leading process nodes to mature node components — such as power management integrated circuits (PMIC), electrically erasable programmable read-only memory (EEPROM), compound semiconductors, and microcontrollers — all of which are critical to the functionality of AI systems and infrastructure.

Chip Innovation Drives AI Development

Although AI may seem like a modern invention, its foundations can be traced back to decades of advances in cutting-edge computing power. However, hardware limitations at the time ultimately constrained the capabilities of AI.

Over the past few years, the increasing scale and complexity of logic, memory, networking, power, and thermal management have paved the way for the widespread deployment of high-performance AI systems.

These technological advances have accelerated the rise of AI data centers. Traditional data centers have existed for decades, used to manage enterprise IT operations, website hosting, and storage, while modern AI data centers represent a fundamental specialization of capability, rather than incremental evolution. Every AI data center server rack integrates a complex array of advanced semiconductor devices, purpose-built to support parallel processing, data locality, and scalability. An AI server rack is composed of approximately 20,000 individual semiconductor die, packaged into more than 4,500 chips. These chips include logic processors that deliver high-throughput computing, ultra-low-latency memory subsystems, power management units, and network components.

Semiconductors account for approximately 95% of the bill of materials value of a mainstream AI server rack. Each server rack in an AI data center contains more than 4,500 chips, which in turn are composed of around 20,000 individual semiconductor die. A mainstream data center can hold more than 45 million chips.

As organizations across industries race to deploy AI-powered solutions, demand for AI data center capacity and, by extension, advanced semiconductors, has exploded. The impact of this surge in demand is felt across the entire semiconductor value chain.

Chip designers are under pressure to shorten innovation cycles and release new generations of cutting-edge devices more frequently. At the same time, manufacturers must make major manufacturing technology upgrades to meet the leaps in performance required for AI workloads. These workloads are pushing the limits of previous generations of hardware, exposing key issues in architecture optimization, thermal management, and data transmission across large systems. To address these challenges, chip manufacturers are co-designing hardware and software, working toward tighter integration of memory and computing, and developing innovative packaging technologies that enable high-density, high-bandwidth configurations.

This is a self-reinforcing cycle of innovation between semiconductors and AI: advances in semiconductor technology and AI systems have spawned an increasingly complex developer ecosystem, which in turn requires increasingly powerful AI systems and semiconductor technology. As the developer ecosystem grows more complex, AI models continue to scale up, requiring more data, faster processing, tighter inter-system coordination, and more computing, constantly pushing the limits of existing chips and systems. This has led to a shift toward greater specialization in semiconductor design, including more efficient and higher-performing processors, more specialized memory stacks, and high-speed interconnects that can support distributed AI workloads. In fact, semiconductor designers and manufacturers are increasingly leveraging AI methods to advance the development of next-generation products.

Throughout this report, we frequently reference two primary AI workloads — training and inference — which represent different stages of AI computing and influence how chips are designed. Let's use a simple example to understand these two workloads more clearly:

Training is the process of teaching a model by exposing it to massive datasets. For example, to build a cat recognition model, you might show a neural network thousands of images of cats and non-cats. Through repeated exposure, the model learns to recognize patterns such as ears, whiskers, or body shape, allowing it to distinguish cats from other objects.

Inference is applying a trained model to new, unseen data. Continuing the cat example, once the model is trained, it can generate a new image of a cat.

In short, training is the process by which the model learns, while inference is how the model applies what it has learned to the real world, such as generating query responses, making predictions, or identifying patterns.

Classification of Semiconductor AI Chips

AI server racks rely on multiple semiconductor technologies working together, each purpose-built to meet the demanding requirements of modern AI workloads (Figure 3).

Inside each server are many specialized chips, including AI accelerators that perform the parallel processing required to run complex AI training and inference models. These chips perform billions of operations simultaneously, enabling efficient processing of massive amounts of information.

Memory semiconductors are critical to supporting this computing layer, enabling fast and reliable data access. As AI models grow in size, the amount of system data has grown exponentially. High-performance memory helps ensure that processors are continuously supplied with data, avoids bottlenecks, and keeps systems responsive. The specialization of memory and logic for AI tasks is becoming increasingly important for next-generation AI system design.

A wide range of power and network semiconductors enable efficient energy delivery and seamless interconnection both inside AI servers and between systems. AI servers run many chips simultaneously, each coordinating closely with others, so fast and reliable management of power, data, and computing resources is critical. At the same time, distributed AI workloads rely on fast, low-latency communication between nodes, so network semiconductors are critical for coordinating computing across groups of interconnected servers.

Detailed Breakdown of the AI Hardware Stack: Chips in an AI Data Server

As described in the next section, a teardown of an AI server rack's hardware reveals a highly modular, vertically integrated system. It is composed of tens of thousands of interdependent semiconductor components in each subsystem within the server, from CPUs and accelerators to signal conditioning chips, voltage regulators, memory chips, and control logic, with no exceptions.

A server rack contains a set of coordinated trays, each designed to deliver a specific function. Each tray (or subsystem) contains a combination of semiconductors and supporting electronic components — all loaded with semiconductor components — that work together to deliver the throughput, energy efficiency, and reliability required for AI workloads. Understanding this layered architecture is key to understanding the strategic importance of the semiconductor supply chain that underpins it. As AI continues to scale, the ability to source, design, and integrate these components will be just as important as computing itself.

Typically, an AI data server rack in a modern data center consists of five types of trays: 1) Compute Trays, 2) Power Trays, 3) Network and Intelligent Platform Management Interface (IPMI) Trays, 4) AI Accelerator Interconnect Trays, and 5) Coolant Distribution Unit (CDU) Trays. Each tray houses different semiconductor components that perform critical roles. The exact composition of trays within a server rack varies from data center to data center, depending on original equipment manufacturer (OEM) choices, form factor, layout, power supply, and the data center builder and/or operator.

This report analyzes the semiconductor components in a general-purpose AI data server tray by tray.

From AI accelerators to individual voltage regulators, every component in the system plays a critical role in ensuring performance, efficiency, and reliability at scale.

Compute Tray

The compute tray is the core of the AI server rack, providing the primary processing logic required for AI workloads. Each compute tray typically contains one or more compute boards, non-volatile memory express (NVMe) solid-state drives (SSD) for storage, network interface cards (NIC) for connectivity, data processing units (DPU) for organizing data into independent processing units, and power distribution units (PDU). This tray holds the most expensive, advanced, and concentrated semiconductor components in the server rack, as it directly determines performance per watt and AI workload throughput.