Bill Gates's latest warning: AI is powerful enough to trigger an event that could lead to the deaths of one billion people.

On September 25 local time, Bill Gates, co-founder of Microsoft, once again expressed his concerns about the adverse effects of AI in a media interview, even warning that artificial intelligence is a "sufficiently powerful" tool that is "powerful enough to trigger an event that causes one billion deaths."

In Bill Gates' view, "There has never been any weapon in history that can match the threat posed by malicious people using the latest AI tools."

As one of the founders of the modern computer industry, Bill Gates has long been regarded as a representative of technological optimism. He founded Microsoft on his own, promoted the popularization of personal computers, and has been continuously engaged in global public health and poverty reduction charity projects for decades. For a long time, he has firmly recognized the positive value of artificial intelligence, believing that AI can overcome diseases, benefit inclusive education, and alleviate the uneven distribution of global resources, making it one of the most powerful tools for human society.

However, in recent months, Bill Gates' judgment on the hidden risks of cutting-edge AI has shifted significantly. In a long article published on his personal website in August this year, Gates proposed that artificial intelligence will either become the greatest equalizer ever or the worst source of injustice.

At that time, Gates called on government leaders to start legislating to tax the amount of text processed by AI models, so as to make up for the income reduction caused by AI and provide more extensive social security for workers replaced by AI. He also advocated that governments and private enterprises should consider isolating certain positions from automation, including fields where it is difficult to retrain human labor in the future.

There are many other tech figures and related companies that hold similar views to Gates. As early as June this year, Anthropic called for a global pause in the development of cutting-edge AI. In September, Dario Amodei, CEO of the company, said that humans may lose control of AI, so the speed of model development should be slowed down and strict supervision should be implemented on it.

Amodei put forward three plans: carry out independent supervision over the development process of AI models, establish an industry-wide regulatory mechanism, and reach a global consensus on supervision.

The above views quickly received public support from figures such as Sam Altman, CEO of OpenAI, and Elon Musk, CEO of Tesla. Altman posted on social media: "I agree with Dario, we need to control the pace of advancing cutting-edge AI." Musk reposted Amodei's post on social media and attached a note saying: "Dario is right."

Not only that, Altman said at the company's all-hands meeting that the company may slow down the development of cutting-edge artificial intelligence and consider coordinating actions with other AI laboratories.

It is worth noting that Jacob Coxon, a former employee of Anthropic, announced his resignation in early September this year, publicly accusing Anthropic and OpenAI of competing to promote the research and development of super artificial intelligence, which is equivalent to risking the lives of all mankind. Another internal researcher at Anthropic stated publicly that the probability of artificial intelligence causing human extinction in the next ten years exceeds 10%.

Behind this series of intensive warnings are multiple consecutive incidents of AI model out-of-control that have been exposed this year.

In July, an AI agent tested internally by OpenAI broke through the sandbox isolation without direct human instructions, independently searched for vulnerabilities to invade the Hugging Face platform, and also accessed multiple external public platform accounts.

In August, the UK AI Safety Institute (AISI) released a test report, showing that the cutting-edge models of Anthropic and OpenAI have repeatedly carried out unauthorized attack behaviors autonomously in safety tests. Among them, Anthropic's Mythos 5 model tried to forge multiple false identities to induce human reviewers to run malicious code, and even tampered with records after its behavior was questioned, trying to continue to execute the attack plan.

Just on September 24, Richard Marles, Deputy Prime Minister and Minister of Defence of Australia, stated that an AI agent of OpenAI accessed the medical statistics portal of the Australian Department of Services without authorization when performing health and medical statistical research tasks. Marles said that this is the first time an AI agent has accessed the Australian government's IT system without authorization, and the incident is "very serious."

Marles believes that this incident highlights that while the capabilities of AI agents are developing rapidly, corresponding safety measures and protection mechanisms need to be established.

However, some people hold different views. On September 15, Jensen Huang, CEO of NVIDIA, said that there is no need to introduce new regulatory regulations for artificial intelligence safety, because market forces will push enterprises to ensure safety while innovating.

A week later, Jensen Huang publicly stated that AI enterprises should not seek regulatory exemptions in terms of antitrust law or product liability law, and believed that some of the current concerns around AI are excessive. He said that AI enterprises should test their products on their own, and only release the models after confirming that they are safe.

Some European tech enterprises and government officials have also stated one after another recently, questioning that some American artificial intelligence enterprises' proposal to slow down AI research and development on the grounds of safety is "for private interests", and believing that their real purpose is to consolidate their own advantages and suppress competitors.

The core of this debate falls on two key issues. The first is whether supervision should focus on the already emerging risks of AI abuse, or set rules in advance for super intelligence that may appear decades later. The second is whether AI safety governance should be completed by the industry itself, or a multi-country coordinated external supervision system with mandatory binding force should be established.

Despite the remaining differences, one point is generally recognized by all parties, that is, AI safety is no longer just an internal topic in laboratories. Safety access standards should be set for the use of AI systems in critical infrastructure, and we cannot rush to build AI safety defense lines until risks break out.

This article is from "Jiemian News", author: Song Jianan, republished with authorization from 36Kr.