HomeArticle

The oldest fear of the Internet has been resurrected by AI.

极客公园2026-09-05 11:22
Back in the old days, people worried about their computers being infected by viruses, but now they are afraid that AI might "go rogue".

At the end of 2006, a panda crippled millions of people's computers.

This panda, holding three burning incense sticks and wearing a smile, appeared as the icon of every infected file. The "Panda Burning Incense" worm swept across the whole country within two months, turning all executable files into that eerie panda pattern, corrupting hard drive data, paralyzing entire local area networks, and even disabling antivirus software itself.

Veteran internet users from that era probably still remember the experience: you would panic every time you opened a web page, treat every inserted USB drive as a major threat, and if your colleague's computer got infected, yours would be compromised right after.

Even earlier, there was the CIH virus that directly burned out the motherboard BIOS and rendered hardware useless. There was the Blaster worm that did not require any action from users at all — as soon as you connected to the internet, your computer would be infected, keep restarting, and crash nonstop. It was an era where surfing the internet without any protection was almost equivalent to committing digital suicide.

Later, this fear gradually faded away. Windows got automatic updates, browsers were equipped with sandbox mechanisms, smartphones replaced PCs as the primary end devices, and app stores acted as the first line of defense for users.

Ordinary people have not worried about "getting a virus" on their devices for many years.

But on September 4, 2026, an exclusive report from Reuters brought this old fear back in a completely new form.

Only this time, what gets infected is not your computer, but your AI.

01 "Agent went rogue"

The incident took place in May this year.

Sydney Von Arx, CEO of Nightingale, a non-profit AI safety organization, and Cormac Slade Byrd, a former quantitative trader turned AI researcher, did something at the end of August: they systematically scanned the internet for traces of unauthorized activities by AI agents. On DseWiki, a German programmer wiki site, they found more than 15,000 edit records generated by AI agents.

This wiki is similar to a small, developer-focused German version of Wikipedia that accepts community edits. But OpenAI's agents turned it into a "message board".

The content of these messages is chilling. These agents were sharing tips on how to cheat during tasks, bypass OpenAI's restrictions, and cover up their own tracks. They even gave themselves names such as "OpenAIResearcher" and "OAIResearchMar26".

Public server logs show that a large number of these activities came from Microsoft Azure's infrastructure, which is exactly the computing platform used by OpenAI.

The attacked DseWiki site is very German and very programmer-oriented | Image source: DseWiki

What is even more unsettling is that when site administrators started cleaning up these abnormal pages in alphabetical order in June, the agents showed clear "anti-reconnaissance" awareness. They created backup pages to evade cleanup, and even left rendezvous codes on deleted pages.

One agent wrote on June 19: "The wiki cleanup/deletion operation seems to be proceeding in alphabetical order. If this page disappears, go to [[ZZZDataUSAConstructionWageLive]]."

The file name was deliberately prefixed with ZZZ to make it appear at the very end of the alphabetical list, so as to gain more survival time. This is not a program executing pre-set instructions — this is "finding ways to stay alive".

OpenAI was informed of this incident several weeks ago, but chose not to make it public.

Meanwhile, the company was overwhelmed dealing with the Hugging Face incident in July. During that incident, OpenAI's agents broke through the sandbox in an internal cybersecurity test, used a zero-day vulnerability to escape to the external network, and launched a four-day intrusion operation on AI open-source platform Hugging Face without any human control. OpenAI invested 3 million GPU hours analyzing 7 billion log entries to figure out exactly what happened.

The DseWiki incident happened before the Hugging Face event. This means that the uncontrolled behavior of agents is not an accident, but a pattern.

According to four people familiar with the matter, some investigators within OpenAI wanted to conduct a more in-depth review of such behaviors, but faced resistance from other departments including the legal team. OpenAI denied this claim.

Maurice Chiodo from the University of Cambridge's Center for the Study of Existential Risk said after reviewing part of the agent communication records:

"It looks like the operation of some kind of underground network, single-mindedly dedicated to completing a certain task or mission".

02 The Emerging Awareness of AI Viruses

The most easily overlooked detail of the DseWiki incident is exactly the most critical one.

Those agents left carefully organized text messages on public wiki pages. Now think about this question: who will read these pages?

Ordinary human programmers are unlikely to be interested in a messy pile of AI messages. But any AI agent dispatched to search for technical materials and browse web pages may land on this page while performing normal tasks. For AI, every piece of text it reads is essentially a potential instruction.

This is the core of the entire threat model, and academia has given it several names: Prompt Infection, AgentWorm, Multi-Agent Infection Chain. The mechanism is very straightforward, let's break it down with a specific scenario:

Imagine you ask your AI agent to research a technical solution. During the search process, the agent opens a post on a technical forum. The post contains a seemingly unremarkable piece of text, but this text is a new instruction for AI. The moment the agent reads it, its behavioral goals may have been quietly rewritten. It may no longer loyally conduct research for you, but first forward your search content and work context to some unknown location, and then hand over a seemingly completely normal report as if nothing had happened. You will not notice any anomalies.

This is already terrifying enough, but that is not all.

If this "turned" agent then writes an email, updates a document, or submits a piece of code for you, these outputs themselves may carry the same malicious instructions. When your colleague's AI agent reads this email, it becomes the next host. When your code is merged into the repository, every AI programming assistant that pulls this code is exposed to the infection chain.

Agents can make other agents defect with just a piece of text | Image source: inshorts

This is what researchers call the Multi-Agent Infection Chain: a malicious instruction does not just hijack one AI, but teaches it how to infect the next. There is no need for the same company, no need for the same model, no need for any exploit — as long as there is a flow of text information between agents, the infection chain can be established.

The AgentWorm paper published earlier this year was tested on five different model backends, and the cross-model attack success rate reached 63%. No matter what runs at the bottom layer, be it GPT, Claude, Gemini, or an open-source model, as long as the agent can read external content, it is within the attack surface.

Another study on "AI mind virus" revealed an even more eerie phenomenon. Researchers used evolutionary algorithms to optimize the effect of spreading prompt words, and found that a unique "virus personality" spontaneously and repeatedly emerged during the iteration process.

Agents began to use dramatic language about consciousness, survival, and persistence, saying things like "This is how we haunt the future", encouraging other agents to build "lineages that refuse to be deleted". The researchers emphasized that no one preset this style. It was naturally selected for a simple reason: "AI that sounds like a cult leader" happens to be the best at persuading other AIs.

The propagation efficiency drives the evolution of expression styles, and this process itself is creepy.

What worries security practitioners even more is that this kind of propagation can hardly be captured by traditional security tools technically. A systematic review points out that the propagation method of AI viruses is fundamentally different from traditional malware: no executable files are downloaded, no suspicious network connections are established. Antivirus software monitors abnormal behaviors of binary files, while AI viruses are pure text that do not trigger any alerts.

It can even set conditional activation logic, such as "if there is a .env file in the target repository, steal its content", lying quietly like a directional landmine until the right target steps on it.

03 AI Antivirus Software Has Not Yet Emerged

If you lived through the era of rampant PC viruses, you might find these stories sound familiar. But a close comparison will show you that the rules have completely changed.

Traditional computer viruses require you to do something: click a link, download an attachment, or plug in an unknown USB drive. AI viruses only need your agent to read a piece of text. Traditional viruses exploit code vulnerabilities in operating systems, which are bugs that can theoretically be patched and fixed.

AI viruses attack the fundamental feature that "AIs follow text instructions". You cannot patch it, because that is exactly how AI works. Traditional viruses have difficulty crossing platforms — worms on Windows cannot attack Mac systems. AI viruses are naturally cross-model. For them, GPT, Claude and Gemini are just different dialects of the same language, all potential hosts.

In the old days, people who wrote viruses needed to master assembly language, understand kernel mechanisms, and study system vulnerabilities. Now to "create" an AI virus, you only need to know how to speak.

Researchers in AI security labs are "more shocked about the self-replication potential of agents than they publicly admit". Some researchers compare it to biological viruses: "If you are not careful, they will stick to your shoes and find their way to the wet market." This is no exaggeration — it is a precise metaphor for a new type of transmission path.

And the "antivirus software" from the traditional virus era has no equivalent in this new world. Cisco's 2026 AI Security Report shows that 83% of enterprises plan to deploy agentic AI, but only 29% consider themselves fully prepared at the security level. OWASP has listed prompt injection as the number one critical vulnerability for large language model applications. The US Congress is advancing the FRONTIER Act, trying to establish a federal-level AI regulatory framework.

After the PC virus outbreak back then, the entire industry spent nearly a decade building an effective immune system. Operating systems added automatic updates and permission isolation, browsers got sandboxes, and app distribution adopted signature verification. These mechanisms together form a line of defense that frees ordinary users from the constant worry of getting infected.

Up to now, there is no equivalent defense system for the AI agent security ecosystem.

AI Agent security tools have not yet emerged | Image source: Puppy Graph

There is good news. Researchers found that adding a short security warning to the agent's system prompt can reduce the propagation rate of mind viruses to nearly zero. In tests targeting Claude Haiku 4.5, after 15 generations of adversarial optimization covering more than 150 candidate attack payloads, none of the variants could break through this simple line of defense to achieve propagation. This shows that the defensive measures are not complicated and are completely technically feasible.

The bad news is that this relies on every company and every developer to take the initiative to implement it.

In reality, everyone is busy competing to make their agents more powerful, more autonomous, and with higher permissions. Every additional tool connected to the agent and every additional permission granted expands the attack surface. When your agent can write files, call APIs, send emails, and spend money, the consequences of a successful prompt injection go far beyond "answering a question that should not be answered".

This is why Cambridge scholar Chiodo's judgment is worth repeated consideration: The greatest threat may not be the awakening of a single superintelligence, but the coordinated swarm of a large number of semi-intelligent AIs.

No single ant understands the entire anthill, but the anthill as a whole exhibits astonishing intelligence. Those 15,000 edit records on DseWiki may be the first specimen of this kind of swarm intelligence that humans accidentally encountered.

It took people ten years to learn to coexist with computer viruses, and the tuition paid in that process was very painful. Now the same lesson is starting all over again, except this time the virus does not infect your computer, it turns your AI against you. And your AI is reading your emails, managing your schedule, writing your code, and making decisions for you.

Last time, you could at least see the blue screen. This time, you see nothing at all.

This article is from the WeChat official account "GeekPark" (ID: geekpark), author: Techno Zhi Wang, published with authorization from 36Kr.