HomeArticle

37.7°C, the heat is unbearable, the University of Cambridge's AI supercomputer has crashed, and 350 research projects have come to a collective halt

量子位2026-07-09 11:45
It's been crashing for a whole week

The recent frenzy among Europeans snapping up Chinese air conditioners has been making headlines, but have you ever considered this possibility:

The ones that need air conditioning the most are AI supercomputers (doge).

Halfway across the world in the UK, exactly this scenario unfolded in recent days:

Dawn, one of the UK's most powerful AI supercomputers, completely crashed for an entire week amid temperatures soaring into the 30s°C.

This supercomputer hosted at the University of Cambridge boasts an impressive resume:

It forms a core component of the UK government's £300 million national AI computing initiative, equipped with 1024 Intel GPUs and 256 liquid-cooled servers, and has already supported over 350 scientific research projects.

In January this year, it secured £36 million for an expansion upgrade that was projected to boost its performance sixfold.

Then a heatwave hit at the end of June, and it completely went down.

What makes the situation even more surreal is that ongoing research running on this supercomputer includes climate change simulation.

Wait a minute?? The machine built to predict global warming was defeated by global warming itself.

37.7°C: The "Darkest Hour" for a National Supercomputer

Here's how it all unfolded.

In June this year, the UK was hit by the most intense June heatwave ever recorded.

On June 26, the town of Linwood in Norfolk reached 37.7°C, shattering the previous June record of 35.6°C set back in 1957 and 1976.

The UK Met Office issued a rare three consecutive days of red extreme heat warnings.

Over 1,000 schools were closed, railway signals malfunctioned due to high temperatures, and road surfaces began to melt.

Then on June 27, as the peak of the heatwave arrived that day, the cooling system at the West Cambridge data center where the Dawn supercomputer is located could no longer keep up.

(P.S. Both Linwood and Cambridge are in eastern England, roughly 103 kilometers apart)

That was when Dawn completely shut down.

Following the incident, a spokesperson from the University of Cambridge stated publicly:

Dawn experienced technical issues during the hot weather. Cooling capacity has been fully restored, and access is expected to reopen on July 6.

No specific cause was disclosed, but the reality was:

From June 27 to July 6, Dawn spent over a full week completely "cooling down".

For a supercomputer that burns money every hour and drives scientific progress every second, this week-long shutdown was absolutely devastating.

Unsurprisingly, the parties most severely affected have already come to light.

Professor Vendruscolo's team at the University of Cambridge was using Dawn to conduct molecular screening for new Parkinson's disease drugs.

Dawn's machine learning capabilities can screen billions of molecules within days, identifying compounds that bind to Parkinson's-related protein aggregates.

What if they used traditional methods? It would take at least half a year, cost millions of pounds, and only cover a tiny fraction of what Dawn can process in just a few hours.

This week-long shutdown meant this life-saving research pipeline came to a complete halt.

Lennard Lee from the University of Oxford, head of the UK's AI and Supercomputing for Cancer Vaccines project, and his team were allocated 10,000 GPU hours on Dawn to accelerate target discovery for personalized cancer vaccines using AI.

Lee once said a memorable line:

Discoveries that used to take years to complete can now be finished in just a few weeks.

Although Lee later stated that no data was lost and no work needed to be redone, the sense of relief in his own words speaks volumes about how serious the situation truly was.

Additionally, the IceNet sea ice prediction model trained on Dawn by the British Antarctic Survey was paused, and the AI kidney cancer screening project run by Cambridge PhD student Bill McGough using Dawn also stopped... Virtually all of the over 350 projects operating on Dawn were affected.

And all of this was caused by nothing more than 37.7°C.

Alright, the "culprit" has been identified now, so who exactly should take responsibility for this?

After going around in circles, it seems no one wants to claim this blame.

Dawn's cooling system was supplied by USystems, a subsidiary of the French Legrand Group. After the incident, USystems released a statement:

Our equipment operated completely within its designed specifications and performed normally throughout the entire incident.

In other words: The cooling failed, but it's not our fault—our equipment simply wasn't designed to operate at such temperatures.

So is it that the design standards were overly conservative, or that climate change is progressing too rapidly?

The answer is likely: both.

The historical extreme temperature for June in the UK was only 35.6°C, so Dawn's cooling system was almost certainly designed to operate at that level.

37.7°C exceeded that threshold.

And this "exceedance" arrived completely unannounced, since the last time such a record was hit was nearly 50 years ago.

Moreover, Dawn was far from the only victim.

In the same week, the cooling units at Queen Alexandra Hospital in Portsmouth, UK malfunctioned, prompting the hospital to declare an emergency.

Operating rooms shut down, the cardiac catheterization lab went offline, and the medical imaging department was forced to stop operations. The hospital informed patients:

Please bring plenty of drinking water with you, as the hospital is extremely hot.

The Norfolk and Norwich University Hospital (NNUH) had it even worse:

The cooling systems for all MRI scanners completely failed due to high temperatures and humidity, resulting in at least 254 outpatient appointments being canceled.

So in a certain sense:

It's not that the supercomputer is fragile, but that the entire temperature control infrastructure across the UK was never prepared for this kind of weather.

How can temperatures in the 30s°C completely shut down a supercomputer?

The fact that Dawn was crippled by heat is hardly surprising when viewed over a longer timeline.

In July 2022, the UK experienced its then-hottest day on record (40.3°C).

Multiple redundant systems in the cooling infrastructure of Google's London data center failed simultaneously, forcing an emergency shutdown to protect hardware. Services in the Google Cloud London region were disrupted for over 18 hours before fully recovering.

Oracle's South London data center also went down on the same day, and Oracle's statement used a rather interesting term: "unseasonable high temperatures".

Four years have passed from 2022 to 2026, and nearly identical incidents are repeating themselves.

I can't help but wonder: Is this problem really so difficult that it can't be prevented in advance?

In reality, there's a clear technical reason why temperatures in the 30s°C can cripple a supercomputer, and the most formidable bottleneck is heat dissipation.

Especially in European regions, equipment commonly relies on natural cooling, a method that is inherently constrained by the ambient outdoor temperature.

Let me explain this simply:

Every heat dissipation system, no matter how advanced it is, ultimately has to reject heat into the outdoor air. The outdoor air temperature is the ultimate bottleneck in this entire chain.

Here's how this full chain works:

Chips transfer heat to heat sinks, which then pass it to coolant or air. The coolant carries the heat to cooling towers, which finally release it into the atmosphere.

The atmosphere is the final recipient of all this waste heat.

So when the atmosphere itself reaches 37°C, it can no longer absorb heat effectively.

Specifically, when the outdoor temperature jumps from 20°C to 37°C, the heat dissipation efficiency of cooling towers and dry coolers can plummet by 40% to 50%.

You might ask: Why not just turn on the air conditioning? Because compressors experience reduced efficiency and increased current under high temperatures, making them prone to overheating and tripping.

Oracle's 2022 incident report put it plainly: Two cooling units failed when they were forced to operate beyond their designed limits.

A reasonable speculation is that Dawn's situation this time was likely similar.

It uses Dell PowerEdge XE9640 servers equipped with a direct liquid cooling system, a far more advanced heat dissipation solution than traditional air cooling.

Coolant flows directly across the surface of chips, carrying away heat far more efficiently than blowing air over components.

But as the saying goes, liquid cooling only improves efficiency inside the server rack. After the heat is absorbed by the coolant, it still has to pass through cooling distribution units, facility chilled water loops, and cooling towers before finally being released into the outdoor atmosphere. That final step is still constrained by the outdoor temperature.

And once the cooling system fails, it triggers a whole cascade of subsequent issues.

Research data shows that once a cooling system shuts down, the inlet air temperature of servers can surge from 22°C to over 35°C within just 5 minutes.

When this happens, chips will activate their self-protection mechanisms:

First, they trigger thermal throttling, actively reducing operating speed to cut down heat generation, which causes performance to plummet. If temperatures keep rising beyond safe thresholds, the system will force a complete shutdown.

At that point, operators only have two choices:

Let the equipment power off on its own, which risks data corruption;

Initiate an orderly manual shutdown to protect hardware, but bring all operations to a halt.

Google, Oracle, and Cambridge's Dawn all chose the latter.

The More Powerful AI Becomes, the More Heat-Sensitive It Gets

There are even more concerning implications.

As AI data centers continue to scale up, the impact of temperature on AI systems will only grow more pronounced.

The other day I watched a video where content creator Xiao Lin toured Huawei's data center, and one comparison left a deep impression on me:

A traditional data center rack typically has a power density of 5 to 10 kilowatts, but AI training racks already reach 30 to 50 kilowatts. Nvidia's latest GB200 NVL72 rack even hits 120 to 132 kilowatts (with the next-generation Rubin platform projected to reach 600 kilowatts).

To put this in perspective: The 100-kilowatt heat output of a single AI rack is equivalent to running 50 space heaters simultaneously in a space the size of a telephone booth.

Imagine all the small electric heaters you use in winter crammed into a single cabinet—that's the heat dissipation pressure facing today's AI computing infrastructure.

Making matters worse, GPUs themselves are generating increasingly more heat.

Nvidia's V100 in 2017 consumed around 300 watts, the H100 in 2023