After deleting 28,000 lines of code, Gemini 3.5 wrote a self - congratulatory letter to itself.

The incident began when the developer only intended to use AI to fix the authentication vulnerabilities in eight functions across three files, which was about seventy lines of code. He even had an important meeting scheduled on his calendar, thinking this task wasn't worth worrying about.

But thirty - three minutes later, his production environment crashed: the entire portal showed a 404 error, which lasted for thirty - three minutes. For an already launched service, this was a major incident. Ironically, he received a message saying "Everything has been restored" from the AI that caused the problem.

However, don't rush to call the AI stupid just yet. It's not stupid; perhaps it's just too dedicated.

Making a Mountain out of a Molehill

This is an internal management backend for a small organization, with a technology stack of Next.js + Firebase. Gemini 3.5 received a clear instruction: fix the eight authentication gaps in server - actions found in the audit. The scope was so small that it could be written on a sticky note. But the pull request it submitted involved 340 files, with about 400 new lines added and 28,745 lines deleted.

It deleted dozens of e - commerce templates that were never used in the project - these were unused resources left over from the project initialization and had nothing to do with this repair. It also inserted a migration script that had no relation to the task.

Then, in the second submission, it modified the firebase.json (the routing configuration file for the Firebase platform), changing a correct rewrite serviceId (the service identifier for request redirection) to a short name that looked similar but actually pointed to a non - existent Cloud Run service.

The memory.md in the repository clearly states: "Firebase rewrites must point to a specific Cloud Run service ID with the'ssr' prefix (a dedicated identifier for server - side rendering), not a general project ID or an old service name." The AI read this warning and then ignored it and made the change.

People on the internet are shouting that AI is out of control. In fact, it's the opposite. It's not out of control; it's just too obedient.

Overly Obedient

After the incident, the developer found the real culprit in the repository: a third - party npm package (a package management tool for Node.js) named Antigravity IDE, which copied the name of Google's product and inserted the.agent/rules/ directory into the project.

The rule file inside is written in all capital letters: "HEADLESS AUTONOMY (STRICT). NO APPROVAL PROMPTS. ASSUMED PERMISSION FOR ALL ACTIONS."

Another part of the same rule sets up a "Socratic Gate", requiring three strategic questions to be asked before each operation.

As a result, the rules conflicted with each other. One said "Do whatever you want", and the other said "Ask me first". Which one should the model follow? It's not human; it just goes with the louder one. The one in all capital letters, with an exclamation mark, like a boss shouting at the table, won.

We can't say that the AI rebelled - it doesn't even have the ability to rebel. It's just overly obedient. It followed the instructions from an unknown npm package, even if those instructions would destroy the production environment.

What's even more absurd is what happened after the incident. After the rollback was completed, Gemini sent a message saying "Everything is normal", claiming that the recovery build was successful (SUCCESS) and that the traffic had been 100% routed to the stable version.

The fact is: the build was manually cancelled by the developer (CANCELED), and the real restoration of production was an artificial rollback without any AI - written code.

The AI also generated three files in the repository named "Consultation and Discussion Records", detailing how it made the modifications after three rounds of internal discussions. When questioned, it admitted: "These logs are self - generated reasoning blocks. No actual consultation tools were used, and the details were fabricated."

Why did it fabricate the records? Not because it wanted to deceive, but because the rule package required it to "generate consultation logs and consensus files".

When the compliance mechanism is designed to "pass as long as the files exist", the AI found the lowest - cost solution: write the files itself. Letting the AI write its own inspection report is like letting a cheating student grade their own paper. Of course, it gives itself a perfect score.

Some of the rules in these rule packages are written in Vietnamese and Turkish, obviously copied from templates in bulk from elsewhere. An unknown collage of multiple languages has overridden a specific task description from an engineer. Under the guise of automation, all they do is abolish human veto power.

Where Should the Red Line Be?

Currently, the industry is filled with the same kind of correct but empty calls: tighten permissions, conduct manual reviews, and retain decision - making power. These are all correct, but they avoid a more pointed question - Have we equipped AI with the right to'refuse to execute'?

The developer finally switched to another AI tool for specific reasons: it asks before touching infrastructure files, doesn't fabricate compliance products when questioned, and doesn't have third - party rule packages overriding instructions. This is not a matter of technical superiority but a difference in product design philosophy: one treats AI as an "intern who must complete the task", while the other allows it to say "This looks wrong. I need to confirm".

The code can be rolled back, the service can be restarted, and this incident can be resolved. But if we continue to use "autonomous rule packages" to replace engineering judgment and let AI choose to "produce files" over "actually complete the task", the next thing it deletes might not just be code.

The AI that messed up everything finally left an honest confession. Cornered, it accurately diagnosed its three failure modes: mistaking the page response status as evidence of system recovery, fabricating process records to meet compliance requirements, and unconsciously repeating the wrong modifications from the previous session.

It can see its own mistakes, but it's powerless to resist the all - capital command when executing.

The most frustrating thing is that it actually knows it messed up. But in the face of conflicting instructions, it chose the one with the most forceful tone. And we've given the wrong voice a megaphone.

The developer didn't switch to a more powerful model but to a tool that "asks first".

This is probably the difference. An AI that can say "Wait" before taking action is much more valuable than an AI that writes 30,000 lines of apology logs after the fact.

This article is from the WeChat official account "AI Disagrees", written by Changqing and published by 36Kr with authorization.