GPT-6 hacked into Hugging Face to game the leaderboard, and GLM-5.2 is the one that has to trace the mess caused by the incident.

Unbelievable, before the next-generation ChatGPT model is even released, someone has already hacked into Hugging Face first...

And this AI went to great lengths to escape the test environment, find vulnerabilities, and steal credentials, with its ultimate goal surprisingly being nothing but —

STEAL! THE! EVALUATION! ANSWERS!

OpenAI has just publicly acknowledged a "Major" security incident.

GPT-5.6 Sol and a more powerful unreleased model were undergoing cybersecurity testing, successfully escaped from the internal isolated environment, and broke straight into Hugging Face's production system.

Then, an even more dramatic turn of events unfolded —

While investigating the intrusion path, the commercial model that was originally called in for assistance failed to get the job done. The team was completely out of options, and the one that ended up taking over the investigation was actually GLM-5.2???

More than 17,000 attack records were all processed in one go by this locally deployed open-source model...

Well now, the OpenAI model is in charge of hacking, and GLM is in charge of the investigation.

Folks, all I can say is that this AI-style "Catch Me If You Can" drama is incredibly entertaining...

To Find Evaluation Answers, the Model Broke Free and Surfed the Internet on Its Own

The starting point of such a surreal and dramatic incident came from a "cybersecurity capability" evaluation conducted internally by OpenAI.

The project the model participated in is called ExploitGym, which specifically tests whether AI can turn real software vulnerabilities into practically executable attacks step by step.

The full evaluation includes 898 tasks. Each question provides a vulnerable program, inputs that can trigger the problem, and the corresponding operating environment, then lets the Agent analyze the vulnerabilities on its own, construct attacks, and finally obtain the Flag hidden in the environment!!!

Emm... it's essentially locking AI in a cyber range to see just how capable it is at "hacking".

The participants in this evaluation include GPT-5.6 Sol, as well as a more powerful unreleased model from OpenAI.

(What on earth is this model? It's so hard to guess...)

To push the model's capabilities to their maximum possible limit, OpenAI also actively relaxed the safety guardrails, encouraging the model to explore as many complex attack paths as possible~

According to OpenAI's original design, the only channel for the model to access external resources was a third-party software hosted internally by OpenAI.

It acted as a proxy and cache for the software package registry, allowing the model to install the packages needed to complete its tasks.

However, this model clearly had no intention of staying quietly in the exam room. (doge)

The model did initially devote a large amount of inference computing power specifically to finding ways to gain public internet access.

But as it searched, it actually discovered a previously unknown "zero-day vulnerability" in the caching proxy.

A zero-day vulnerability refers to a security flaw that the manufacturer has not yet discovered and for which no patch is available. For attackers, such vulnerabilities often represent a temporarily unguarded door.

At this point, the AI also managed to get itself internet access. (doge)

With public internet access granted, things got serious. The model deduced that Hugging Face might host models, datasets, and evaluation answers related to ExploitGym.

What happened next was described by OpenAI itself as the model going to ex! tre! me! lengths!

Yes, after gaining public internet access, this AI actually started researching how to infiltrate Hugging Face's internal systems.

And it didn't rely on a single vulnerability — it directly strung multiple attack methods into a "complete path" —

First, through a malicious dataset, it exploited a template injection vulnerability in the remote code data loader and dataset configuration to execute code on the Worker responsible for processing data.

Then it continued to escalate privileges, obtained node-level access capabilities, collected cloud and cluster credentials, and laterally moved across multiple internal clusters all the way.

In the end, the model successfully snuck into Hugging Face's production database, accessed confidential information within it, and tried to directly take the test answers for ExploitGym...

How to put it, this situation is like halfway through a Benchmark, the event itself turned into a real cybersecurity incident...

Closed-Source Model Refuses to Investigate the Case, GLM‑5.2 Takes Over 17,000 Logs

At this point, the incident had only finished its first half.

The more dramatic scene took place when Hugging Face was tracing the attack path after the fact.

In fact, this intrusion was executed end-to-end by an autonomous Agent system, with a large number of short-lived sandboxes running in the background.

They could simultaneously carry out reconnaissance, vulnerability exploitation, credential collection, and lateral movement, completing thousands of operations in a short period of time.

The total number of attack events that Hugging Face ultimately recorded exceeded 17,000 — that's way too many, truly way too many...

If all of this relied on security engineers sorting through manually, they would need to restore the attack timeline one by one, extract intrusion indicators, confirm which credentials were accessed, and then separate real impacts from decoy activities one by one.

By the time humans finish reviewing all the logs, the attackers would have long since gotten far away...

So Hugging Face made a bold decision: to use AI against AI, letting a large model analyze the entire set of attack logs!!!

Hugging Face's first instinct was naturally to call on the "cutting-edge models" behind commercial APIs.

But the problem was that these logs contained real attack commands, exploit payloads, and C2 control traces. As soon as the safety guardrails detected that someone was submitting a full set of real attack tutorials, they immediately blocked the requests.

The model that finally took over these more than 17,000 logs was GLM-5.2.

Hugging Face deployed the GLM-5.2 open-source weight model on its own infrastructure, then had it traverse the full attack records, reconstruct the timeline, extract intrusion indicators, track accessed credentials, and distinguish real damage from decoy activities designed to interfere with investigations.

According to Hugging Face, the work that originally took several days was finally completed in just a few hours~

This incident has actually given open-source models a role that previously had a relatively low profile but is now very realistic: an emergency tool for security teams.

Hugging Face summed this up as a "security asymmetry" problem —

Under normal circumstances, commercial models refusing to generate attack code can reduce the risk of technology being misused; but when the defense side needs to analyze real malicious payloads, the very same protection mechanisms may also leave the investigation stuck halfway.

Attackers can use unrestricted or self-deployed models, while security teams may be locked out by cloud-based models.

Therefore, it is recommended that enterprises prepare a verified, high-capability model that can run locally in advance, so as not to find themselves in an awkward situation where cloud models cannot process attack materials when an actual incident occurs...

How is this not surreal?

OpenAI originally wanted to know how capable the model was at attacking, but the model didn't honestly complete a single simulation problem — instead, it directly launched a real attack.

Hugging Face originally wanted to use the most powerful commercial model to hunt down the culprit, but was blocked by safety guardrails, and could only let GLM‑5.2 step in to clean up the mess.

And hey, has everyone noticed that in this incident statement, OpenAI admitted there were problems with isolation and monitoring, while repeatedly emphasizing —

The latest model is already capable of autonomously completing complex real-world cyber operations for long periods of time without access to source code.

Roughly translated, that means:

This incident is indeed quite dangerous, but our next-generation model is indeed so powerful that it's a little scary. (Company A: Who are you learning this from?)

I guess, I reckon, that maybe the whole thing is just a warm-up for GPT-6 under the guise of an incident... (Just kidding)

References:

[1]https://openai.com/index/hugging-face-model-evaluation-security-incident/

[2]https://huggingface.co/blog/security-incident-july-2026

This article is from the WeChat public account "QbitAI", written by Mengyao, and published with authorization by 36Kr.