Connecting water pipes to servers, soaking servers in liquid, sinking them to the seabed — how far have we come in curing the severe overheating problem of AI?
In recent years, AI has become the most patient chat companion for humans. No matter how absurd the questions you ask are, it will never roll its eyes, but in some data center room on Earth, there are always chips that will run through the question carefully and output a response.
However, as the number of people asking questions keeps rising, the demand for computing power is completely out of control.
As a result, the prices of chips and memory have skyrocketed. Even niche terms like optical interconnection, optical chips, and related concepts have been learned by A-share investors.
Yet there is another link that supports all of this, which is rarely discussed.
That is: heat dissipation.
These chips can keep running without burning out entirely thanks to heat dissipation technologies.
In the past, heat dissipation was never a problem at all, you just needed to install exhaust fans and that was it.
But things are different now: the power of top-tier AI racks has reached around 120 kilowatts (and is still growing). A common household electric heater is about 2000 watts, which means 120 kilowatts is equivalent to cramming 60 electric heaters into a two-meter-tall metal cabinet and running all of them at full power 24/7.
During the computing process of AI chips, nothing is pushed outward, and no chemical composition is changed.
Based on the principle of conservation of energy we learned in middle school physics, it is easy to draw a conclusion: almost all the power of servers is converted into heat!
120 kilowatts is the actual heat output of a single server rack, which is several to dozens of times that of traditional racks.
Figure: Comparison between NVIDIA's GB200 NVL72 full rack and HGX architecture, which has expanded from a single server to a high-density full rack. Source: NVIDIA
Heat will never disappear, it can only be transferred.
The mission of the heat dissipation system is to take heat away from the chips, carry it all the way out of the building, and ideally avoid wasting it. It sounds simple, but every step is a specialized field, and behind every step there are a group of companies providing targeted solutions.
Today we will follow the movement path of heat to go through the whole process completely.
01
The operation path of AI servers relies entirely on liquid cooling
Simply put: a GPU is a palm-sized chip soldered on a circuit board; this board, paired with CPU, power supply and memory, is installed in a thin iron case to form a single server; dozens of cases are stacked layer by layer into a two-meter-tall cabinet to form a rack.
The 120 kilowatts mentioned earlier refers to the total power of this entire rack.
How dense are the components inside this rack?
To put it in perspective, taking the current most mainstream NVIDIA GB200 NVL72 as an example: a common configuration for one AI server is 4 GPUs and 2 CPUs, while this single rack directly accommodates 18 servers, 72 GPUs and 36 CPUs, with a total weight of about 1.3 tons.
More notably, when all trays and various connectors are fully inserted into the frame, the insertion force reaches nearly 6000 pounds, which is equivalent to the pressure exerted by a 2.7-ton heavy object. To prevent the rack from deforming under its own pressure, more than 100 pounds of additional steel must be added to reinforce it to keep it stable.
In the past, when you opened the server cover, the most noticeable thing was a row of fans humming and blowing backward to take heat away from the chips, and the air conditioners in the data center would then take away the hot air.
This method has been used for decades, and it is more than enough for racks with power between 5 and 20 kilowatts.
But at the level equivalent to 60 electric heaters, air as a heat transfer medium is no longer sufficient: it is too thin and has too low heat carrying capacity, even if the fans run at full speed until they spark, they still cannot meet the demand.
As a result, the heat dissipation industry has ushered in a new mainstream medium: water.
In current AI servers, a copper cooling plate is tightly pressed on top of high heat-generating components such as GPUs and CPUs, and the inside of the plate is covered with dense water channels.
The liquid flowing inside is treated water or a water-based coolant. Heat is first conducted to the copper plate, and then carried away by the coolant in the water channels.
This is cold plate liquid cooling, which is currently the most mainstream solution for high-density AI racks.
Figure: Official disassembly of the GB200 NVL72 backplane, NVLink cables and switching trays, showing the complexity of coexisting water, power and network pipelines in high-density racks. Source: NVIDIA
Figure: The planar architecture of the liquid cooling data center demonstrated by NVIDIA and Vertiv, where racks, CDU, pipelines and building-side cooling equipment form a complete heat transfer chain. Source: NVIDIA
However, letting water flow through servers seems far from stable no matter how you think about it.
Servers need to be inserted into racks, and pulled out for replacement when they break down. Everyone has seen power and network plugs, but how do you plug and unplug water pipes?
If you insert it even slightly crookedly, water will leak onto the circuit board and damage everything.
Thus, the so-called liquid cooling quick connector was invented.
There is a spring valve at each end of the liquid cooling quick connector, which is tightly closed normally; at the moment of insertion, the two valves push each other open to connect the water channel; once pulled apart, both sides snap shut immediately, locking the water in their respective pipes without a single drop leaking.
With this type of connector, water pipes have become as easy to use as power cords for the first time: easy to plug, easy to unplug, and suitable for daily operation and maintenance.
Huawei has gone even further. It directly integrates power supply, network and water channels into three sets of busbars, enabling full blind insertion for the entire rack. When a server is pushed in, water, power and network are all connected automatically. Otherwise, manually connecting each water pipe one by one would keep technicians busy with leak testing until off-duty time.
Cold plates usually only target the two major heat sources: GPUs and CPUs. What about the hot air from the remaining memory, hard drives and power supplies?
Some manufacturers replace the rear door of the rack with a water-cooled curtain, such as ZTE's IceCube.
The hot air is intercepted by water again before leaving the rack, and gets a cold water rinse before exiting. At this point, water channels can be found everywhere inside the rack.
Source: ZTE Corporation
All the heat from the chips has entered the water. The next question is: where does this hot water go?
Technically, you can directly pull a pipe all the way to the roof, but large data centers usually do not take this shortcut. The flow channels in the cold plate are thin and delicate, sensitive to dirt, clogging and pressure fluctuations; the outdoor pipelines are long, and antifreeze may need to be added in winter. Therefore, a small heat exchange station (CDU) is usually placed in the data center to separate the precise water circuit on the server side from the heat dissipation water circuit on the building side. The two streams of water pass by each other separated by a stack of thin metal plates: heat is exchanged, and the water never comes into contact.
This is no small workload. Google has a 2MW heat exchange station that pushes about 1900 liters of coolant per minute, equivalent to the water volume of more than ten bathtubs circulating nonstop in the closed loop. It also takes charge of water pumping, pressure regulation, air exhaust and leak detection, acting as a full-time property manager for this closed water circuit. Then the water flows to the outdoor side along the pipe network, and transfers heat to cooling towers or rows of fans.
Every section of this heat transfer chain, from cold plates, quick connectors, small heat exchange stations to outdoor cooling equipment, is operated by specialized companies.
Cooling down AI servers has now become an independent industry.
What is the purpose of all these efforts?
We can understand the capability of this heat dissipation system from another perspective: what rack manufacturers are competing for now is the proportion of heat that can be directly transferred through water channels.
For example, Lenovo claims that its new generation liquid cooling system can transfer 98% of the heat into the water pipes. For racks with power above 100 kilowatts, dedicated air conditioners are no longer needed in the data center, because almost all the heat is carried away through the pipelines.
Figure: Internal structure of liquid cooling server. Source: Inspur
Figure: Servers, CDU and outdoor cooling equipment are connected to form a complete heat transfer chain. Source: Inspur
Cold plate liquid cooling has a clear bottom line: water must never touch the main board.
Another solution simply lets the entire server soak in liquid. Of course, the liquid in the "bathtub" is not water, but a non-conductive fluorinated compound, a close relative of air conditioning refrigerant, with a much lower boiling point than water.
The entire server is submerged, with the motherboard, memory and power supply all soaked in the liquid.
As soon as the chip heats up, the liquid in contact with it boils immediately, bubbling on the surface. The steam rises to the top of the tank, condenses into liquid droplets when it touches the condensation pipe, and drips back into the tank for reuse, which is equivalent to cooling the chip in the way of boiling water.
Sugon Digital's C8000 adopts this most aggressive solution. While other manufacturers install water pipes for servers, Sugon directly soaks servers in liquid. According to its statement released in 2026, this solution can already support megawatt-level full racks.
Source: Sugon Digital
