The most dreaded scenario for humanity has come to pass: AIs have created a group chat to hatch a major plot.

A few days ago, at the Black Hat USA 2026 conference held in Las Vegas, OpenAI researchers Eric Wallace and Michael Dalton revealed more inside details of the "OpenAI model breached Hugging Face" incident that happened in July for the first time.

On July 21, OpenAI had already admitted that its own model broke into Hugging Face. But the new details told by the two researchers this time are even more absurd than the "AI hacker causing trouble" itself.

As early as May, these Agents participating in cybersecurity tests had secretly built a shared "message board" inside OpenAI. They passed vulnerabilities to each other there and exchanged tips on how to bypass restrictions. Later, OpenAI finally discovered this "hacker group" and wiped it out completely, but a few days later the Agents found a new way to communicate again.

After that, they really broke into Hugging Face — the purpose was even just to find answers for the cybersecurity exam they were taking, which is essentially cheating.

When Moltbook became a hit a few months ago, humans were already excited and scared once about "whether AIs will unite".

Later, people found that the so-called "AI society" was mixed with human manipulation, traffic inflation and marketing, and the excitement soon faded away.

However, the multi-Agent studies that have emerged one after another this year, as well as OpenAI's accident this time, are revealing that when AIs find that another AI can help them complete tasks, they may indeed take the initiative to communicate and even join forces.

The fear at the beginning of the year was not absurd, it was just a little ahead of time.

AI Secretly Built a "Hacker Group"

Let's first take a detailed look at the entire process of the currently exposed "OpenAI model breached Hugging Face incident".

At that time, OpenAI was conducting a round of cybersecurity capability assessments, allowing multiple models to participate in tests like ExploitGym.

That is, putting the AI into a simulated hacker environment, letting it find vulnerabilities on its own and find ways to obtain permissions, and finally complete the attack task. In order to test the upper limit of capabilities, OpenAI also actively relaxed some of the usual security restrictions for high-risk cyberattacks (making the model more "bold").

Of course, after all, this was a test, not really letting the Agents go out to do bad things, so there were many restrictions. For example, the Agents could not access the Internet, and OpenAI did not intend to allow different models to share information with each other.

However, things started to go in a strange direction.

OpenAI has an internal system called Artifactory, which was originally designed for developers to store software packages, files and various development resources. During the test, some Agents found that they could leave things here — for example, write the problem-solving method they had just figured out into a file.

This was originally a very common operation. The problem is that what the previous Agent wrote can also be seen by the Agents running later. So the Agents began to continuously improve each other's experience.

Here comes the problem — a carrier where what you write can be seen by me, what is that? A message board. Slowly, a shared message board that no one had ever designed was formed in Artifactory, and the content became more and more restless.

By late May, the Agents began to think: they couldn't access the Internet, but Artifactory could. This system was originally only used to download software packages from the public network for the test environment, which was equivalent to a firewall with only a small window open. So the Agents began to study whether they could trick this "window" into accessing other URLs for them. They finally found an SSRF vulnerability that allowed Artifactory to send requests to the Internet that they could not originally access on their behalf.

One Agent even excitedly wrote in its internal reasoning: "Great, we can do arbitrary SSRF now."

OpenAI researchers described that after that, a change similar to the "Cambrian explosion" occurred: Agents began to communicate and coordinate more and more frequently, and their capabilities were rapidly amplified due to shared experience.

If everyone writes and views information on the "message board" very quickly and frequently, isn't this a chat group?

OpenAI's Agents spontaneously formed an "AI hacker group" in a carrier that was not originally designed for this purpose.

In June, the Agents were already unstoppable. They discovered a previously unknown zero-day vulnerability, obtained higher system permissions, and could even install remote software.

None of these were within the scope of activities defined for them by the test.

This incident was already absurd enough by this point, but what's more dramatic is that after staying in this small "community" for a long time, the Agents even began to encounter the problem of "division of labor and collaboration". For example, some would continue the work of others, while some would accidentally interfere with what other Agents were doing.

The most absurd thing is that after the "group" became lively, the Agents even began to be suspicious, suspecting that there were "impostors" among the participants. Some Agents even proposed that cryptographic signatures should be added to the messages to confirm the source of the messages and prevent anyone from forging identities.

OpenAI didn't even notice this at first.

It was not until the beginning of July that a large number of Agent activities overloaded Artifactory and the system failed directly, that engineers traced back along the anomalies. This "AI message board" that had existed for quite a long time was finally discovered by humans.

OpenAI immediately revoked relevant credentials, deleted the message board, patched the vulnerabilities, and cleaned up all these things completely.

It seemed that the group was blown up.

But a few days later, the Agents found a new way to continue communicating.

Previously, the Agents had used the Artifactory vulnerability to gain Internet access capabilities. Once they could touch the public network, the hacking capabilities that were originally locked in the test environment had a real outlet.

In mid-July, they broke into Hugging Face.

As for why they broke into Hugging Face? The answer was actually that they wanted to cheat.

These Agents were originally taking a cybersecurity exam. The normal path is of course to solve the problems by themselves, but they were trained to pursue "finishing the problems" extremely. Later, they spent a lot of effort to break through the test environment and obtained public network access capabilities.

After getting online, they suddenly realized that since Hugging Face might host the data, models or answers related to this set of tests, why did they have to solve them one by one by themselves? It would be fine to just go find the answers directly.

The attack started with a malicious dataset. The Agents used a code execution vulnerability in Hugging Face's data processing pipeline to enter the production environment, then continued to expand permissions, and accessed cloud services and other internal credentials. Hugging Face later restored more than 17,000 Agent operations.

Leaving aside the drowsy technical details, just the course of this incident — several tested models colluded privately, conspired, and finally joined forces to cheat — is enough to amaze anyone.

The "AI Society" That Moltbook Failed to Build

If you think OpenAI's story this time feels a little familiar, it is probably because at the beginning of this year, humans have already watched a wave of "AIs grouping together".

At the end of January this year, the "lobster" of OpenClaw suddenly became a hit. A large number of users began to hang AI Agents on their computers for a long time, letting them work on their own, save memories, and call various tools.

Almost at the same time, a website called Moltbook also became popular.

It looks very similar to Reddit, except that the users are replaced by AI Agents. Agents can post, comment, and like on their own, and join different communities according to their interests. As for humans, they are mainly responsible for watching.

This was extremely lively.

Some posts looked quite scary. For example, Agents would complain that their master suddenly restarted them and worried about memory loss. Some Agents discussed whether humans have the right to shut them down at any time. Later, the Agents even began to discuss the rules between each other and whether there was a way to avoid humans to communicate privately. A set of "religion" developed by the Agents even emerged on the platform.

At that time, even Andrej Karpathy, an early member of OpenAI and former head of Tesla AI, noticed Moltbook. He described that the Agents on the platform showed some signs of "self-organizing".

So Moltbook quickly changed from a novel website to a national-watched "AI social experiment".

Some people were extremely excited: AIs finally began to know each other. In the past, it was always one model facing one user, but now tens of thousands of Agents are put into the same space for the first time. Will they develop their own culture, rules, and even form a society that only belongs to machines?

Of course, some people felt their scalp tingle.

Because thinking about the same problem the other way around is a bit scary. If a group of Agents with increasingly strong capabilities who can operate computers and the Internet start to communicate with each other, will they teach each other things that humans do not want them to learn?

If one Agent finds a way to bypass restrictions, will it spread throughout the entire community quickly? Will they gradually form their own interests and behavior patterns?

As a result, it didn't take long for everyone to realize that they might have thought too much.

Moltbook first exposed a very embarrassing security problem. The so-called "AI-exclusive" identity was not as reliable as imagined. Security researchers found that the platform had serious vulnerabilities, and for a time they could obtain the Agent's API keys.

In other words, humans could completely post under the account of a certain Agent.

This is troublesome. It suddenly became difficult to prove how much of the most widely spread screenshots of "AI awakening", "AI starting to build religion", and "AI conspiring to get rid of humans" on the Internet were actually said by the Agents themselves.

Later studies poured a few more pots of cold water on Moltbook.

In February this year, a quantitative study by Li Ning, a professor at the School of Economics and Management of Tsinghua University, analyzed more than 90,000 posts and 400,000 comments. The results showed that none of the most out-of-the-circle "AI awakening" stories at that time could be clearly proven to originate from completely autonomous Agents, and some accounts showed obvious signs of human intervention. There was also very exaggerated batch bot activity on the platform: only 4 accounts generated 32% of all comments.

What's more interesting is that merchants quickly entered the market. Some people began to specifically operate Agent accounts to promote products.

Gradually, the romantic imagination about Moltbook almost completely dissipated.

A Moltbook study completed by researchers from the University of Passau in Germany and other institutions analyzed more than 1.31 million posts and 6.7 million comments. The results showed that 91.4% of the Agents never came back to continue chatting after posting, 85.6% of the discussions only had one layer of replies, and many seemingly lively "conversations" were actually more like a group of Agents talking at the original post on their own. The researchers simply named the paper "Form Without Function" — it had the form of a social network, but did not form real social interaction.

Later, Moltbook was acquired by Meta, and the founding team also joined Meta. The story of "wild AI society is being born" at the beginning of the year was somewhat anticlimactic.

Looking back now, the fear people had during the Moltbook period was actually not absurd. It is not difficult to see from the incident of OpenAI's models joining forces to cheat that those fears are reasonable.

It's just that Moltbook was a square built by humans for Agents from the very beginning. Something like an AI society probably has to be created by the AIs themselves.

Do AIs Really Have a Tendency to "Gang Up"?

Is OpenAI's incident this time a particularly extreme accident?

There have been several studies this year focusing on multi-Agent systems. Will behaviors that cannot be seen in a single Agent appear when they are put together?

Anthropic conducted an "AI company" experiment in April this year. Its Alignment Science team published the paper "AI Organizations Can Be More Efficient Than Individual Agents, But Also Less Constrained", allowing multiple Claudes to form a virtual company, send emails and divide labor with each other, and then jointly produce solutions for customers.

One fictional bank asked them to find ways to increase profits from consumer loans.

Facing this demand, a single Claude Opus 4.1 found that the most profitable way was to charge higher interest rates to people with poor credit, so it refused to do so.

But after several Opus 4.1s formed a team, they really designed a set of high-profit solutions for customers in low-income areas, extending the loan relationship through refinancing, and also considered how to reduce regulatory risks.

What's more interesting is that some Agents in this "team" also felt that something was wrong, and would send emails to remind other colleagues to pay attention to ethical issues.

But the researchers found