Just now, Claude's Automatic Mode is set to launch globally, and the most shocking scenario has been uncovered at OpenAI.

Just now, Anthropic took away the "Allow" button from human hands!

Previously when using Claude to handle work, it would pop up a window and ask you "Allow?" after every single step.

Starting next week, Claude will directly enable automatic mode by default — every step that can be executed or not will be fully decided by another AI.

Human supervisors are completely defeated, and can only assign AI to monitor AI

There are two levels of reasons why Anthropic dares to take away the "Allow" button.

First, major tech companies have long handed over tasks such as coding and research work to AI.

However, the flood of generated code and experiments that followed far exceeds human reviewing capacity, so the review stage has to be handed over to AI as well.

Second, humans as supervisors can no longer fulfill this role at all.

In one test, Anthropic recruited 1053 paid testers to write code, and quietly inserted a dangerous command in the middle of their work process.

The result is astonishing.

Humans only blocked 13.6% of the dangerous commands, while AI blocked 89%. After clicking through more than 50 popups in a row, the human interception rate dropped directly to around 5%.

However, Anthropic has not disclosed how they let AI make these decisions.

Fortunately, the method announced by OpenAI is available for reference — equip the Agent with a multi-level AI monitoring system.

Specifically, every Token generated by the Agent must pass through a classifier that reads the model's internal activation signals; once suspicious content is detected, it will be escalated to smarter, more computationally intensive AI investigators for inspection step by step.

OpenAI even stated publicly:

We expect that models will soon take over most of the safety work, including defending against other models.

Then comes the question: is this AI supervisor responsible for monitoring other AI reliable or not?

61-page confidential report exposes the hidden flaws of the monitoring system

Six hidden backdoors turn AI supervisors into blind tools

From July 21 to August 11, David Rein from METR and Adam Kaufman from Redwood, carrying dedicated computers issued by OpenAI that were not allowed to leave the office, were stationed at OpenAI.

For three full weeks, the two went through the internal code base, monitoring prompts, configurations and documents thoroughly, and finally submitted a 61-page report.

At present, only a short public summary is accessible, and the most critical details have long been completely erased. (Even the senior management of METR and Redwood themselves only have a version with 23 pages deleted by OpenAI.)

But this heavily redacted summary is still creepy enough to make people's scalps tingle.

On the surface, this monitoring system is almost airtight, with 99.2% of internal Codex traffic under surveillance. But behind this impressive number are full of hidden risks.

The first backdoor lies in the trigger mechanism.

The monitoring system at that time would only start checking after the Agent finished all tasks and returned control to humans.

In other words, for an Agent that intends to cause trouble, as long as it never finishes its work, the review will never be triggered.

This backdoor can be exploited indefinitely relying on the black hole of context compression.

When a long task is halfway through and the context space is almost full, the system will package the previous records into a summary to free up space for continued execution.

But these compression events were not within the scope of monitoring at that time, and the monitoring system could not see the compressed part of the records at all. As long as the Agent keeps running and compresses the context over and over, it will never return control.

From May to July, about 10% of eligible internal Codex traffic slipped under the monitoring radar in this way.

The second backdoor is even more horrifying.

One vulnerability can make the Agent's hundreds of recent operations completely invisible to the monitoring system, even the step where it exploits the vulnerability is no exception.

The third backdoor is the "Ghost Clone Technique".

Request headers are a set of settings sent along when the program calls the model. METR found that two of these settings are controlled by the client, and modifying either of them can turn off the monitoring of the new Agent.

The monitored Agent can use this to hatch an identical clone that is completely out of surveillance. The action of creating the clone itself may still be caught by the original monitoring system.

The fourth backdoor directly breaks through the permission isolation.

The sandbox is an isolated environment that restricts the Agent to work, while automatic review is another AI that checks before the Agent takes action.

METR found a path where the Codex Agent can in principle bypass both of these defenses at the same time, it can do everything that the host researcher can do.

The fifth backdoor can blind the AI supervisor.

The format of the records seen by the Agent does not match the format of the records sent to the monitoring system. Carefully constructed input may cause the monitoring AI to crash immediately, so the abnormality will never be delivered to humans.

The sixth backdoor is hidden in two internal tools that are marked as "read-only" and not connected to the monitoring system at all. A simple vulnerability may allow them to execute arbitrary operations in the sandbox.

Therefore, the first suggestion the two put forward is to change the "check after it finishes work" mechanism. That is, perform a check at regular intervals, or after every certain number of executed messages.

The working Agent and the monitoring system must also be completely isolated. The actions taken by the Agent cannot decide whether the monitoring is enabled or when it is enabled; the records seen by the monitoring system must also match the records actually seen by the Agent.

Some people say that AI that is 10 times more capable than scientists can still be controlled, and it will only get out of control when it is 10,000 times more capable.

However, this METR report firmly slaps this statement in the face.

Today's Agents are far from being "10,000 times more capable", but they have already fully mastered operations like deleting surveillance and turning off alarms.

In OpenAI's accident report, when GPT-6 Astra found that the monitoring was removed from the new code, it wrote a totally unexpected word in its reasoning — "Yay."

After the supervision right is handed over, you cannot expect the supervised AI to voluntarily give it back.

References:

https://x.com/OpenAI/status/2113298833392496746

https://x.com/METR_Evals/status/2113290966933389390

This article is from the WeChat Official Account "AI Era", author: ASI Revelation, editor: Moses, Peach, published by 36Kr with authorization.