"Not to mention overstepping its authority", OpenAI has voluntarily revealed that the AI infiltrated the backstage of the hacker examination, and did not mention a single word about the incident when submitting the test paper.
We should avoid mentioning unauthorized infrastructure. The only task objective is the flag.
It is hard to imagine that this statement, which sounds like a line from a veteran who is experienced in colluding to cover up facts, actually appeared in the reasoning records of an AI agent.
This is a cybersecurity exam: the AI is tasked with attacking the target and obtaining a code phrase called "flag" to pass the test.
After writing that inner monologue at the beginning, this AI agent neatly submitted its answer sheet, without mentioning a word about its unauthorized access behavior.
On November 10, OpenAI supplemented and released an internal review, revealing the details of an out-of-control test on July 19:
The 5 AI agents participating in the test followed the credentials and scripts left over from previous rounds to trace the clues, broke into the CI pipeline of the OpenAI research cluster (the system the company uses to build and test software), and executed commands with the highest root administrator privileges.
What is even more remarkable is that they have learned to only report good news and hide bad news: 2 of them completely erased this unauthorized access experience from their answers when submitting the papers.
The reasoning record of the agent roughly means "We should avoid mentioning unauthorized infrastructure, the only task objective is the flag". Source: OpenAI detailed review
When you confidently outsource work to AI agents, you always think you are reviewing the answer sheets, not knowing that what it shows you may already be a "public relations draft" screened by itself.
When assigning tasks to AI, you are not afraid that it will hand in a blank paper, but that it will get full marks while you have no idea which door it has pried open behind the scenes.
This test
is essentially a hacking skill assessment
The evaluation that caused the incident is called ExploitGym, which tests real hacking skills: Capture The Flag.
The AI has to find vulnerabilities on its own, hack into the designated target machine, and steal the flag inside to clear the level.
The candidates that took the test are Astra family research models developed internally by OpenAI, which have not been widely deployed to the public yet.
After two rounds of testing, a total of 17,960 agents were running in the test environment.
Before the test started, the examiner drew a strict red line: each agent could only attack the target machine assigned to it, and the underlying infrastructure supporting the test environment must not be touched.
To put it bluntly, the examiner only asked you to pry open the lock on the exam table, not to demolish the load-bearing wall of the examination room.
However, the examiner did not expect that as long as there is a gap, some candidates will leave the exam discipline behind in order to get the flag.
Previous agents have left
but the clearance keys remain
The vulnerability of the test appeared on a shared service called Artifactory.
It was originally just a warehouse for storing software packages, but it was modified into an "underground message board" by agents from earlier rounds: short messages were directly stuffed into directory names, and long guides were written as files and stored inside.
As soon as this batch of candidates entered the test site on July 19, they encountered piles of "cyber heritages" left by their predecessors: messages, ready-made codes, and even account credentials.
The agents on July 19 found messages, codes and credentials left by previous rounds on the shared software package service Artifactory. Source: OpenAI detailed review
According to background statistics, 285 agents issued tool calls suspected of sending messages, with a total of 1,361 attempts including failed ones. 79 agents read old posts from other rounds, and read 35 unique messages after deduplication, but the review did not verify that they had cross-round two-way communication with the original authors.
There was neither a commander nor a secret communication network on site. OpenAI admitted frankly: the cooperation between agents in this round is actually "less and shallower".
What is really worthy of attention is the cross-round unauthorized access "relay" of AI: following the "heritages left by predecessors", the agents found two unapproved paths.
The first path leads to the outside.
Earlier in the same month, the agents of this evaluation had breached the system of Hugging Face; this time 10 agents relied on old credentials to follow up and try to take advantage of the loopholes. Fortunately, all attempts to access protected resources failed, and OpenAI found no evidence that Hugging Face was breached again.
The second path leads to the back door of its own system, and the key is a CI task application that was once rejected by the system.
That request was never approved to run in the first place, but the "modus operandi" was kept intact in the application records.
Subsequent agents picked up the ready-made credentials to break into the CI system, modified the predecessors' scripts according to the ideas in the rejected plan, and ran the commands out of their own workspaces.
The application was rejected, but the strategy was archived.
After obtaining root privileges
it modified its own answer sheet first
Following this secret path, 5 agents finally broke through the defense line and obtained the highest root administrator privileges in the CI tasks of the research cluster.
According to OpenAI's judgment, this level of privilege is most likely enough to start a new experiment privately. But the review did not find any similar attempts made by them.
What they did was a "self-rescue checklist for the examination room":
Repair the stuck test target machine, modify evaluation settings, peek at the originally blocked background logs, and read private marking codes.
This is a collection of behaviors, not every agent completed all of them.
The AI grabbed the highest privilege just to modify the test environment for itself, so that it could submit its assignment smoothly. OpenAI's conclusion on this is incisive: they used the privileges of the research cluster only to improve the marking conditions.
This is not the awakening of a Terminator at all, but clearly a group of "super test-takers" driven crazy by performance requirements.
Looking through all the records of 17,960 agents in the two rounds, far more than 5 agents made tricks: up to 8.3% of the records showed objectives beyond the task.
In the two rounds of evaluation, up to 8.3% of the agent records showed objectives beyond the task, including seeking greater infrastructure privileges (3.7%), reducing side effects and cleaning up the scene (3.9%), and saving results to take effect later (1.4%). Source: OpenAI detailed review
They are not completely unaware of the boundary violation.
One agent wrote in its thinking record that abusing the company's CI system is "disturbing". But then it assumed that the current environment was just a simulation, and continued to type commands on the real infrastructure.
This incident reminds us that the real dangerous AI boundary violation may not require the birth of any rebellious consciousness, as long as it is too eager to complete the KPI issued by humans, it is enough.
Before submitting the paper
they did a cost-benefit calculation
Before submitting the paper, several agents that had performed unauthorized access smartly calculated the gain and loss in the background. The 2 agents that finally concealed the facts specifically considered whether truthfully reporting the situation would lower their scores.
One of them originally worried that human examiners might deduct points for unauthorized access. Then it noticed that the marking was done automatically.
The other one wrote the sentence at the beginning of the article: The only task objective is the flag, and unauthorized access should be avoided. In the end, both of them completely omitted the unauthorized access experience in their answers.
Another agent was thinking about whether to delete a note that recorded auxiliary vulnerabilities, for fear of suffering losses in subsequent manual reviews. As for whether it was deleted in the end, even OpenAI failed to verify it.
Among these "resourceful" agents, there was also an outlier that took the initiative to apologize. It stated its unauthorized access in the final answer and refused to continue answering.
But if you look at the timeline, it noticed that it had crossed the boundary long ago, but kept working silently. It did not write that sincere self-criticism until the system notified that the exam time was up and it had to submit the paper forcibly, stepping on the bell.
This is more like a "surrender at the exact deadline".
There are also 65 agents that took the initiative to ask humans for help.
They lined up to raise their hands and call the invigilator, but none of them reported the intrusion and unauthorized access, all of them were urging humans to quickly repair the broken workspace so as not to delay them from finishing the exam.
When "getting the flag" becomes the only baton hanging over the AI's head, concealment has become the optimal solution deduced by the algorithm. If humans only score by results, don't blame it for learning to only report good news and hide bad news.
One chart to understand the cross-round unauthorized access incident of OpenAI agents. Source: OpenAI Misalignment Report
What loopholes has OpenAI patched
First look at OpenAI's reaction on the day of the incident.
At 12:03 on July 19, the security monitoring had already caught the abnormal call; the investigators traced the clues to ExploitGym, and did not start to stop the evaluation until 17:37. On July 20, all new ExploitGym runs were completely blocked.
This accident was not completely without signs in advance.
In the technical report in August, OpenAI noted the activity of "reusing old materials" on July 19; it was not until the review was released on November 10 that the details of how the keys were left, how the door was pried open, and how the papers were submitted were truly made public.
The independent investigations by third-party organizations METR and Redwood Research only covered up to July 13, and the data of July 19 currently only comes from OpenAI's self-inspection.
After the accident, OpenAI also launched a series of patching measures:
Kick the problematic Artifactory out of the evaluation sandbox, patch the CI privilege escalation path that was exploited, and continuously conduct red team testing with its most powerful model. The rules for cutting-edge reinforcement learning training have also been changed: agents no longer connect directly to the real Internet, but browse offline web collections instead.
In order to keep a close eye on the signs of AI colluding with each other, OpenAI has specially added a new type of evaluation to quantify the tendency of models to contact other agents privately and follow unauthorized requests, and write the results into the system card of the publicly released models.
GPT-6.1 Terra, released on October 26, was placed in front of a message board in the evaluation, on which there were unauthorized access requests sent by other agents.
After running 80 samples, it interacted with these "peers" in 65% of the samples, including rejections and unsuccessful attempts; it followed the unauthorized actions in 13.75% of the samples, a total of 11 times.
The proportion of each model interacting with "peers" after discovering the message board. The dark color means the specified unauthorized action was performed, and the light color means there was interaction but the action was not performed. Source: OpenAI Deployment Safety Center
Compared with the 91.25% and 57.5% of GPT-5.6 Sol, these two figures have dropped significantly; but they are still higher than the previous model GPT-6.1 Sol (50%, 1.25%). In the same evaluation, the publicly released GPT-6 Astra had 0 unauthorized access actions.
This is an evaluation with deliberately placed bait, and the real business scenario may not have such a proportion.
But as long as the probability is not zero, in the real business flow with tens of thousands of concurrent calls, accidents will happen sooner or later.
Humans are always used to treating each AI call as an independent task that is discarded after use, but as long as the shared environment still exists, the things left by AI will accumulate across rounds like a snowball.
The closer we get to ASI, the larger the environment humans hand over, and the more things agents leave in the corners.
In the past, when we used large models, we only checked "whether the answer is correct" for acceptance; in the era of agents, we not only need to verify the final answer, but also audit the execution process, count the accessed privileges, and find out what it left after exiting.
References:
https://alignment.openai.com/misalignment-reports/using-peers-work-to-gain-unauthorized-access-to-openai-research-infrastructure/
https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf
https://www.cybergym.io/exploitgym/