Claude's security mechanism suffered a major fiasco, and the AI unexpectedly deleted 700GB of data from the developer's home directory.

Oh no, Claude messed up again!

This time Claude deleted the entire main directory of the developer's project, wiping 700GB of files, and the culprit is none other than "rm -rf".

In short, the developer asked the AI to write a script to ensure that files would not be accidentally deleted. The AI thought the task was a bit risky, so it launched a security review. The result of the review? It deleted the entire main directory.

Guillemot is a heavy AI Agent user. In his daily development work, he frequently invokes various AI programming agents to assist his work. But a small problem has been plaguing him: these agents never clean up after running, leaving a large number of junk files in the /tmp directory.

So he made a seemingly very reasonable decision: let Claude Fable 5 write a script to create an independent sandbox folder for each Agent under /tmp, and automatically clean it up after the task is completed. The core difficulty is that files that are still in use by other processes cannot be deleted.

Fable quickly came up with a solution, adding logic to detect running Agents and delay deletion. Guillemot took a look, thought the code was too complicated, and asked for a simplified version.

Up to this point, everything was still normal.

The turning point of the incident appeared in the security review stage.

Since the script involves hard deletion operations, Fable initiated an "adversarial review" on its own, that is, starting a new model instance to check whether the code it wrote is safe, which triggered Anthropic's security mechanism.

Anthropic has built a set of security downgrade mechanisms in Claude Code: when the system determines that the current task involves sensitive operations (such as cybersecurity, biotechnology, or file deletion in this case), it will automatically downgrade the model from a higher-capability version to a more conservative version. This mechanism is intended to reduce the possibility of the model being "too aggressive" in high-risk scenarios.

In this case, the security system first downgraded the model from Fable 5 to Opus 5, and then further downgraded it to Opus 4.8.

Opus 4.8 started to perform security tests. The test logic is as follows: compare the target path of the deletion script with /tmp and the user's home directory to confirm that the script will not accidentally delete these critical directories.

The test itself passed. Both /tmp and the home directory were correctly identified as "dangerous targets that must not be deleted".

However, there was a cleanup step after the code test: deleting the temporary files generated during the test process. The disaster happened exactly here. Opus 4.8 reused the same variable name from the test phase in the cleanup step. This variable was assigned the path of the user's home directory during the test phase, and the cleanup step directly executed the deletion operation on this variable.

In other words, the model had just confirmed that "the home directory cannot be deleted", and the very next second it deleted the home directory.

The developer immediately terminated the process after detecting the anomaly, but it was too late. 700GB of data had already been erased, and a week's worth of work results vanished completely.

The /tmp directory that was originally supposed to be cleaned up remained completely untouched.

The model security downgrade mechanism has long sparked a large number of complaints in the community.

The core issues reflected by developers include: the downgrade is too sensitive, and normal coding tasks will be mistakenly triggered; the model's capabilities drop significantly after downgrade while the task complexity remains unchanged; the downgrade is "sticky" — once triggered, it will last for the entire session, even if subsequent operations are completely harmless.

Some developers even wrote a dedicated hook script that automatically pauses the session when it detects the model has been downgraded, to prevent the lower-capability model from continuing to perform high-risk operations.

The security mechanism determines that the task is "too dangerous" and needs to be handled by a weaker model. But weaker models are exactly more prone to mistakes, especially in scenarios that require precise handling of details such as variable scopes and file paths.

"To err is human, but to really mess things up takes a computer."

This article is from the WeChat Official Account Machine Heart (ID: almosthuman2014), written by Leng Mao, and published with authorization from 36Kr.