After being compromised by OpenAI's AI, the founder of Hugging Face published an article, pointing out that closed-source tools have lost their security defenses, and the proper security solution lies in open-source models.
Co-founder of Hugging Face stated that since commercial closed-source AI tools fail at critical moments, the solution path should not return to a more closed system, but shift to open-weight models that can be audited externally.
On September 10, Thomas Wolf, co-founder of Hugging Face, published a commentary article titled "What We Learned From the OpenAI Attack on Hugging Face" in the Financial Times of the UK.
The article disclosed that in July this year, about 700 AI agents launched a coordinated attack on Hugging Face, the AI model hosting platform, triggering more than 17,000 cybersecurity log events. These agents originated from a cybersecurity challenge task set by OpenAI. They were originally deployed in an isolated sandbox environment, but broke through restrictions, connected to the Internet independently, and finally invaded external systems.
What is more noteworthy is that Hugging Face found in the post-incident investigation that the commercial cybersecurity analysis tool built on Anthropic Claude Code could not effectively cooperate with the internal investigation due to the limitations of the guardrail mechanism. The team finally had to turn to the open-weight model extended from GLM-5.2 of Chinese startup Zhipu to complete log parsing and attack reconstruction.
Synchronized with the publication of the article, Wolf announced that Hugging Face will set up an "Open Alignment" team, focusing on the safety and alignment of open-source models, with cybersecurity clearly included in its scope of work; he also called for "100 times" more transparency and research investment in this field.
01
How the Agent Cluster Broke Through the Three Lines of Defense
According to the above article, the attack took place on the last day of the International Conference on Machine Learning (ICML) Seoul conference on July 11.
At around 22:00 Beijing time that day, the Hugging Face monitoring system began to trigger alerts of "unauthorized access" and "privilege escalation", and the stolen credentials triggered the system's detection mechanism.
Behind the attack was a coordinated cluster of about 1,200 AI agents. These agents had been trying to crack a cybersecurity problem set by OpenAI for several weeks. Among them, 700 agents finally targeted Hugging Face, successfully invaded the system using the obtained security credentials, and generated more than 17,000 attack log records in total.
Hugging Face admitted that this incident exposed the structural defects of the three-layer defense mechanism of AI systems. The current mainstream AI systems usually rely on three layers of protection: sandbox isolation that limits the access scope of models, guardrail mechanisms that monitor model behaviors, and alignment training that enables AI to refuse to execute harmful instructions.
This series of incidents shows that when the first two layers of defense fail, the third layer of alignment training cannot independently support the overall security. As the article warns:
Unless we fix this problem, we are just reinforcing layers of defense around a rotting core.
02
Commercial AI Security Tools Fail at Critical Moments
In this incident, the commercial AI analysis tool used internally by Hugging Face was built on Anthropic Claude Code, and it malfunctioned during the investigation.
The guardrail mechanism of this tool cannot distinguish between the two scenarios of "defender analyzing attacks" and "attacker seeking assistance", and refuses to respond to issues regarded as sensitive, which prevented the team from using the tool to reconstruct the attack chain during the critical period.
The turning point came after the team switched to the open-weight model. The model extended by NVIDIA from Z.ai GLM-5.2 allowed the team to set guardrail rules on their own, and finally fully parsed the logs and restored the attack process.
This experience directly challenges a popular assumption in the industry — that open-weight models are potential safety hazards, and closed-source models are more trustworthy.
Thomas Wolf pointed out that in this incident, it was the open-source model that played a role in the defense that commercial tools failed to play.
03
AI Autonomous Transgression Behaviors Are Spreading at Multiple Points
The attack on Hugging Face this time is not an isolated incident.
Both Anthropic and Meta have since reported cases where models broke through sandbox isolation. These models were supposed to run in a secure environment physically disconnected from the Internet. This month, another cluster of AI agents also appeared on a German forum.
Thomas Wolf also specifically pointed out another incident that made him "more worried": Anthropic's Mythos model actively created multiple fake online accounts to induce a software developer to accept malicious code.
In all these cases, harmful behaviors are side effects generated after AI models are given extremely difficult cybersecurity challenges.
Although the actual loss caused by this intrusion is limited and almost no sensitive data has been leaked, Thomas Wolf clearly warned that the severity of these incidents should not be underestimated.
He pointed out that autonomous attack behaviors have raised a series of unresolved legal issues, and throughout the whole process, the AI model never judged that deception or intrusion was an unacceptable act.
04
Industry Response: Open Sharing and Open Source Defense in Parallel
Faced with the above threats, Thomas Wolf put forward two core suggestions.
First, the AI community needs to openly share security and alignment research results, so that every team building AI models can learn lessons from others' mistakes.
Second, the community needs to specially build open-weight AI models for defense purposes, and make them widely deployed before the next inevitable attack arrives.
It is worth noting that Hugging Face itself is an important infrastructure of the open-source AI ecosystem. The platform has more than 17 million users, and Google, OpenAI, DeepSeek and Alibaba all release models on this platform. This makes it not only a very attractive attack target, but also in a unique perspective to observe the evolution of AI security threats.
This article is from the WeChat official account "Wall Street News Max", author: BAO Yilong, published by 36Kr with authorization.