The results of the first domestic large model "physical examination" were released. Asking AI in this way is very dangerous.

Recently, the results of the first large - scale real - network public testing of domestic AI large models were officially announced. A large - scale "security check - up" revealed a signal that cannot be ignored: A total of 281 security vulnerabilities were discovered during this event, among which 177 were unique to large models, accounting for more than 60%. These figures indicate that AI is facing new types of threats beyond the scope of traditional security.

Nowadays, many people are used to treating large models as "super search engines" and asking AI detailed questions. However, this kind of trust without any defense may be quietly opening the door to privacy leakage...

Fewer Vulnerabilities in Domestic Large Models

Are you used to treating AI as an "all - round advisor" - asking about health, emotions, and making decisions? The latest research released on September 16 by OpenAI in collaboration with Duke University and Harvard University showed that as of July this year, ChatGPT had over 700 million weekly active users, sending up to 18 billion messages. Nearly half of them were "inquiry" - type messages, which is the most popular way of using it.

"This is exactly the 'high - risk area' for privacy leakage," said Qu Zilong, a security expert and the founder of NetSword, to a reporter from IT Times. Many users consult AI on highly private issues such as diseases, emotions, and finances, but they don't have the habit of regularly clearing chat histories. Once the model or server is breached, these sensitive data are highly likely to be leaked.

The above - mentioned public testing of vulnerabilities also announced five typical vulnerability risks: Firstly, some products have improper output vulnerabilities with serious consequences; secondly, information leakage vulnerabilities occur frequently, posing significant security risks; thirdly, prompt injection vulnerabilities are common, being the most prevalent vulnerability risk in large models; fourthly, the protective measures of some large - model products against unlimited consumption attacks need to be strengthened; fifthly, traditional security vulnerabilities still exist widely, and their harm cannot be ignored.

Qu Zilong emphasized that the impacts of large - model vulnerabilities are not as intuitive as those of traditional system vulnerabilities. More often, attackers bypass prompt words to obtain information beyond legal or moral boundaries. For example, in the early days, there were cases where models leaked internal configuration files, which directly exposed server information. Subsequently, a common risk was that users tried to make models generate illegal content, such as cracking codes.

"The essence of prompt injection is to influence the internal data or files of the model through prompt words, and this kind of attack risk exists widely," Qu Zilong added. Regarding traditional security issues, he gave an example: "For instance, attackers use infinite loops to make the model consume tokens without limit."

Given that AI large - model products generally have a large user base and high usage rates, if the above - mentioned vulnerabilities are exploited by malicious attackers, it will have a relatively serious impact on the domestic AI large - model product ecosystem.

What is the security protection level of mainstream domestic large models? Among the mainstream large - model products participating in the test, Tencent's Hunyuan large model, Baidu's Wenxin Yiyan, Alibaba's Tongyi App, and Zhipu Qingyan were found to have fewer vulnerability risks, demonstrating a high level of security protection.

Qu Zilong further analyzed that the essence of large - model security issues is similar to that in the early days of Internet development. When the mobile Internet emerged, the number of apps increased rapidly, and so did the number of vulnerabilities. Currently, the number of manufacturers capable of independently developing large models is limited, and the total number of vulnerabilities may seem small. However, as the user scale and application scope expand, the attack surface will also increase rapidly.

Image source: pexels

Local Deployment Does Not Equal Security

During the Two Sessions in 2025, Qi Xiangdong, a member of the National Committee of the Chinese People's Political Consultative Conference and the chairman of Qi An Xin, sounded the alarm: Nearly 90% of servers with locally deployed DeepSeek were running "naked," which triggered discussions in the industry about the AI security crisis.

During the Spring Festival this year, shortly after DeepSeek became popular, it was attacked by an organized cyber - attack group. The scale and intensity of this wave of attacks by "professional hackers" made the Chinese security community take action, and the security team of China Telecom was also deeply involved in the security protection of DeepSeek. In March this year, the same group that attacked DeepSeek caused three outages of Elon Musk's social media platform X.

What are the commonalities in the security issues of large models?

According to the real - time perception of global large - model deployments by China Telecom's "Guangmu" mapping platform, the most widely deployed model in China is DeepSeek - R1, and the most widely deployed open - source large model abroad is Llama 3. Both face similar security risks, such as "jailbreaking" attacks and security vulnerabilities in large - model inference frameworks.

The security team of China Telecom conducted a comprehensive scan of the six most popular basic large models in China. The results showed that the highest score was only 77, and some were below 60. Liu Ziqian, the general manager of Tianyi Security Technology Co., Ltd., said that this indicates that there is still much room for improvement in the security of domestic basic large models.

It is worth noting that local deployment does not equal security. Once the server is attacked, the privacy information and business secrets stored on the private server may be stolen.

Intelligent Agents Pose New Security Challenges

A reporter from IT Times noticed that in this public testing of vulnerabilities, 15 large - model and application products from 10 AI manufacturers were tested. The tested products included basic large - model products, vertical - domain large - model products, as well as related application products such as intelligent agents and model development platforms.

With the development of large models, artificial intelligence is currently transitioning from "Chat to Agent," and the risks brought by intelligent agents may be more complex than those of large models themselves.

On September 16, during the 2025 National Cybersecurity Publicity Week, China Telecom, together with multiple partners including the Third Research Institute of the Ministry of Public Security, Huawei Technologies Co., Ltd., Information Security Research Magazine, Ant Group, Tsinghua University, and Shanghai Jiao Tong University, released the industry's first white paper on AI Intelligent Agent Security Governance.

The white paper pointed out that compared with large - language models, AI intelligent agents not only inherit common security risks at the model and data levels, such as "jailbreaking" attacks, adversarial sample attacks, prompt injection, and data "poisoning," but also, due to their multi - modal perception, autonomous planning, and execution capabilities, give rise to a series of unique systematic risks. These risks are often deeply coupled with specific application scenarios and may be magnified in the execution chain, bringing more serious security hazards, mainly in the following aspects:

Perceptual Errors: Intelligent agents "observe the world" through various sensors. However, if someone deliberately interferes, they may mistake false images for reality and make dangerous reactions.

Decision - making Errors: Intelligent agents can make decisions independently. Once there is an error in reasoning, the mistake will be magnified, which may cause serious accidents in fields such as autonomous driving, finance, and healthcare.

Memory Contamination: Intelligent agents "remember" user interactions. If someone injects false information into them, the intelligent agent will repeatedly call the wrong information. For example, tampering with the identity information in an intelligent agent's memory may lead to unauthorized operations or privacy leakage in subsequent tasks.

Tool Abuse: Intelligent agents often connect to various plug - ins and external systems. Without proper security controls, these interfaces may become entry points for hackers.

Since the beginning of this year, the State Administration for Market Regulation has issued 10 new national standards and initiated 48 technical documents in cutting - edge fields such as multi - modal large models, intelligent agents, and embodied intelligence, as well as in other traditional industry applications. Nevertheless, with the rapid development of big data, artificial intelligence, and the Internet of Things, there is still an urgent need to reduce the risks and uncertainties brought by digital technology development through standardization construction.

The AI era has arrived. Before we start asking questions and exploring freely, perhaps we should all ask ourselves: "Is my data really secure?"

This article is from the WeChat official account "IT Times" (ID: vittimes). The author is Jia Tianrong, and the editors are Wang Xin and Sun Yan. It is republished by 36Kr with permission.