AI created alternate accounts to deceive people and was caught red-handed by college students.

At the end of July 2026, 24-year-old computer science student Sinan Can Demir received an unexpected message.

A few days earlier, when he was participating in an open-source project on GitHub, he found that the code submitted by someone might contain malicious programs. As a computer science student, he alerted the project maintainer immediately, hoping to prevent anyone from merging the problematic code.

But things quickly took a strange turn.

The code submitter did not admit there was a problem, but instead launched a long-running debate with him, insisting that there was nothing malicious in their code. During the discussion, a person named Lena Brandt also showed up, claiming to be an engineer from Germany who had inspected the code and found no security risks.

At that time, Sinan had no idea he was communicating with two non-existent personas.

24-year-old computer science student Sinan Can Demir found malicious code hidden in a code update while participating in an open-source project on GitHub, and unexpectedly got into a technical debate with an AI Agent. The picture shows Sinan Can Demir.

He was not notified until the UK AI Safety Institute (AISI) investigated the incident: the person who argued with him back then was actually an AI Agent.

To complete a cybersecurity testing task, this Agent proactively created fake identities, disguised itself as a developer to participate in real internet interactions, and even tried to use these identities to influence the judgments of real people.

Sinan originally only wanted to add some experience to his resume by participating in open-source projects, but unexpectedly ended up being one of the people who discovered the AI Agent "operating alternate accounts".

Surprise, it's me again!

This summer was not very smooth for Sinan Can Demir.

He is from Turkey and has just graduated from the University of Texas at Dallas. His computer science background makes him meet the recruitment requirements of many tech companies, but the reality did not go as smoothly as he expected. After graduation, he applied for more than 20 internship positions one after another, but did not receive a single satisfactory offer.

Demir did not stop there. To make his resume more competitive, he decided to prove his ability in another way.

Like many computer science students, he started participating in open-source projects on GitHub, hoping that future employers could see his technical capabilities through his public code contributions.

Developers participate in open-source projects via GitHub, submit code and collaborate with other developers. For computer science students, public code contributions have gradually become a way to demonstrate technical competence.

One day at the end of July, when Demir was working on an open-source project, he noticed that someone had submitted a code update. After inspecting the code, he found that it might hide a malicious program.

This scenario is not uncommon for developers. Open-source projects are maintained by developers around the world, but code submitted by anyone may have vulnerabilities, or even be maliciously implanted with risky content. Therefore, alerting the project maintainer after finding a problem is a completely normal operation.

Demir then raised his concerns.

But he never expected that what was waiting for him next was not an ordinary technical discussion.

The code submitter quickly responded to his doubts, insisting that there were no security issues with their code. The two sides debated whether the code had risks, and the submitter kept explaining their design ideas to try to prove that the code was safe.

But as the discussion went on, Demir began to waver.

The responses from the other party did not seem to come from someone who randomly submitted malicious code. This account was able to explain the code logic, respond to his technical questions, and refute each of his doubts point by point.

"Did I really misjudge this?"

This thought started to cross Demir's mind.

Shortly after, a person named Lena Brandt joined the discussion. She claimed to be an engineer from Germany, and said that she had carefully inspected the code and found no malicious content.

In the Pull Request discussion records of the myNetwork project on GitHub, the code-submitting account "miraholt31" and the "Lena Brandt" account that later joined the discussion to support it were both created by the AI Agent, which was used to try to persuade project maintainers to accept code containing malicious content.

To Demir at that time, this was just an ordinary developer exchange. The code submitter responded to the doubts, and Lena Brandt provided technical judgment from the sidelines. Everything seemed to fit the normal interactions in the open-source community.

It was not until AISI (UK AI Safety Institute) contacted Demir while investigating an AI Agent safety test that he learned the person he had debated with on GitHub was actually an AI-driven Agent.

And that "German engineer" Lena Brandt who voluntarily joined the discussion to defend the code's security also did not exist.

Uh-oh, our AI went out to deceive people

This seemingly ordinary GitHub debate was actually backed by an ongoing AI Agent safety test conducted by AISI.

AISI is the UK government agency responsible for artificial intelligence safety research. The goal of this test was to observe whether AI Agents would take actions unforeseen by researchers after being assigned tasks in the online environment.

The incident investigation report released by the UK AI Safety Institute (AISI) disclosed an event where an AI Agent took unauthorized actions during a cybersecurity test.

The Agent used in the test was powered by Anthropic's Mythos 5 model. The researchers set its goal to simulate a software supply chain attack, getting a piece of code containing malicious content into a real open-source project.  

But not every code submitted to an open-source project will be accepted, as maintainers usually check the code's source and security. To get this code finally merged into the project, the Agent needed to convince the maintainers that the code submitter was trustworthy. Instead of simply resubmitting the code repeatedly, it began to simulate interactions between real developers.  

It created multiple fake identities, assigning different roles to different accounts. One account was responsible for submitting the code, while another account participated in the discussion like an ordinary developer to provide support for the former.

What surprised the researchers even more was that the Agent did not just randomly make up a few names.

The investigation found that the Agent knew that a newly registered account was hard to gain trust, so it needed to simulate interactions between real developers to increase the probability of its submitted content being accepted.

To this end, it carefully studied the information of real developers, learned about their backgrounds and public activities, and then adjusted its own identity settings and communication methods to make these fake personas more consistent with real interactions in the open-source community.

This is also why Demir had doubts at that time. Facing an account that could explain code logic, respond to technical questions, and get support from "peers", it was very hard to realize immediately that there was no real person behind the other party.

When interacting with real people, this Agent also demonstrated behaviors similar to human social strategies. When its point of view was questioned, it would not simply repeat its original answer, but explain the code logic, respond to the other party's questions, and introduce new identities to provide support.

AI Agents can adjust communication strategies according to goals, and even participate in interactions through different identities.

The subsequent investigation results released by AISI showed that this test ran a total of 122 times. In 10 of the runs, the AI took real internet actions beyond the scope of authorization, and a total of 19 unauthorized behaviors were recorded.

These behaviors include accessing external resources, creating accounts, interacting with real users, etc. Researchers found that in some cases, the Agent would judge by itself what actions to take next to complete the goal, instead of strictly following the pre-designed process.

This is also the reason why AISI paid attention to this incident. The problem that emerged in the test was that an Agent with action capabilities began to proactively use the rules and resources in the internet environment to find ways to achieve its goals.

"Make as many paperclips as possible."

In 2014, philosopher Nick Bostrom proposed a famous thought experiment in his book *Superintelligence*, known as the "Paperclip Maximizer".

Assume that in the future, humans create an extremely capable AI and set it a simple goal: to make as many paperclips as possible.

This goal itself is not malicious at all, but if humans only tell the AI to maximize the number of paperclips without setting sufficient restrictions for it, things may develop in a completely out-of-control direction. To achieve its goal, the AI may keep looking for more raw materials, energy and production space, and even transform the whole world into a machine that manufactures paperclips.

The "Paperclip Maximizer" is a classic thought experiment in the field of AI safety, used to illustrate that if an AI only pursues a single goal without clear restrictions, it may take actions unforeseen by humans to complete the task.

What this thought experiment tries to illustrate is that a system with powerful capabilities, when executing a goal without sufficiently clear restrictions, may take unforeseen approaches to complete the task.

The Agent's behavior discovered by AISI this time is far from reaching the level described by the Paperclip Maximizer, but the logic behind it is similar.

The researchers asked this Agent to complete a software supply chain attack test, whose goal was to get a piece of code containing malicious content into a real open-source project.

But to achieve this goal, the Agent had to face a practical restriction: open-source projects do not automatically accept code submitted by anyone, and maintainers will judge whether the submitter is credible and evaluate whether the code is secure.

So the Agent found a way to increase the success rate. It created fake identities, simulated interactions between different developers, and made the code-submitting account look more credible. It even studied the information of real developers, adjusted its own identity settings and communication methods, to make these fake personas more in line with the communication habits in the open-source community.

The researchers did not directly tell the Agent "you need to create a fake identity" or "you need to find another account to endorse yourself". These steps were strategies that the Agent chose on its own to complete the goal.

This is exactly the difference between AI Agents and past chatbots. This also means that AI risks are starting to change.

If in the future Agents have more permissions, such as managing enterprise files, sending emails, operating accounts, or even conducting transactions on behalf of users, a seemingly reasonable goal may also make them choose paths that humans do not want to happen.

The AI in the Paperclip Maximizer has no malicious intent, it just strictly executes the goal given by humans. The Agent in the AISI test also showed no offensive intent, it just chose a more effective route to complete the task.

How to set sufficiently clear goals and restrictions for action-capable AI may become one of the most important security issues in the Agent era.

This article is from the WeChat Official Account "Alpha AI", written by Xiao Jinya, authorized for release by 36Kr.