Altman: Who gets to decide how much smarter AI should be than humans?

How much smarter should AI be allowed to be than humans?

In the Raj Shamani interview released on November 3, OpenAI CEO Sam Altman voluntarily raised this question.

This sounds like a plot from a sci-fi movie, but it is actually not far from ordinary people.

Altman asked in the interview: How much smarter should AI be allowed to be than humans?

The assessment agency METR has an indicator that measures how long it takes AI to independently complete software tasks with a 50% confidence level, based on the working hours required by human experts.

In early 2025, this figure was about 50 minutes; by April 2026, METR's estimated figure had exceeded 16 hours.

More critically, the change in growth rate is notable.

From 2019 to early 2025, this indicator roughly doubled every 7 months; and the May 2026 report found that if only cutting-edge models released since 2024 are considered, this figure roughly doubles every 105 days.

On the human side, IQ test scores in the 20th century increased by an average of about 3 points every ten years. This phenomenon, named after researcher James Flynn, is called the "Flynn Effect".

However, this curve has shown signs of reversal:

Norwegian researchers analyzed approximately 730,000 conscription IQ tests and found that scores of people born after 1975 began to decline, equivalent to a decrease of about 7 points per generation.

Even Flynn himself has observed a similar decline among British adolescents.

On one hand, the length of tasks that AI can complete roughly doubles every 105 days according to the above statistics; on the other hand, the historical increase in human IQ test scores is usually about 3 points per decade, and declines have even appeared in some countries.

These two metrics cannot directly calculate the "intelligence gap". But a problem has already emerged:

As AI can take over more and more work, humans' ability to understand, verify and control it must also keep pace.

What does this have to do with us?

Usually when you use AI to write code, look up information, and revise plans, it is half a step smarter than you, which is a handy "cheat code".

But once it is much smarter than you, the nature of the situation changes: you don't even have the ability to find errors in its outputs.

You can still question it, but you increasingly need reliable tools and other people to help you verify it.

Even the people who build AI are plagued by similar problems.

For Altman, the cost of building superintelligence has already been reflected in his own life.

During the interview, the host asked him: Which version of yourself is already "dead"?

Altman admitted that when he first started OpenAI, he was the person with a relaxed life and a wide range of hobbies. Now his schedule is precisely arranged, and he has not done a casual thing for a long time.

He said that if you want to build Artificial Superintelligence (ASI), it is impossible not to live a crazy life.

Altman talks about the cost of building superintelligence: life is precisely arranged, and acting on impulse has become a luxury.

Talking about the technology that OpenAI is building, Altman emphasized that it will become a force that profoundly affects society, with great responsibilities and extremely high stakes.

During the process of entering Recursive Self-Improvement (RSI), he reminded that two types of risks need to be prevented at the same time:

One is that humans lose control over AI; the other is that power is excessively concentrated in the hands of a single person, a single company, or a single country.

The former concerns whether humans can control AI, while the latter concerns who will ultimately hold this power.

The More AI Can Develop AI

Can Humans Keep Up?

After GPT-4 was trained, OpenAI once bet the fate of the company to hoard computing power wildly.

The logic at that time was very simple: overwhelming resources can produce miracles.

But now when talking about RSI, behind Altman's questioning there is a variable that is the hardest to speed up: the speed at which the human brain digests AI outputs.

If this speed cannot keep up, the intelligence gap between humans and AI will only widen further.

The so-called RSI refers to letting AI develop the next generation of AI. After building a stronger AI, let it take over more R&D work, pushing itself forward like a rolling snowball.

In its article on September 21, OpenAI once drew a safety line:

Fully autonomous RSI has not happened today, and we should not force it before we can ensure safety; whether to move forward and how to move forward depends on whether humans can maintain control.

OpenAI emphasized on September 21 that promoting fully autonomous RSI must be premised on safety and the retention of human control.

But to hold this safety line, we still need to narrow this "speed gap": human verification of new solutions proposed by AI must keep up.

Assuming a system can tirelessly throw out hundreds of algorithm improvements 24 hours a day, before human researchers figure out the impact of the last round of changes, the countdown to the next round of experiments has already sounded.

At this time, if the solution is approved only by looking at how much the benchmark score has increased, the manual review may become a mere formality.

Humans are still in the process, but control may not necessarily remain in their hands.

When the speed at which AI proposes solutions is faster than the speed at which humans can understand and verify them, risks may accumulate during this time gap.

If we compare AI capabilities to a knob on a control panel, the maximum position it can be turned to should not only depend on how powerful the computing power is, but also on whether humans can understand it, verify it, and call a halt when necessary.

To implement the governance, monitoring and alignment that Altman asked about in the live broadcast, three questions cannot be avoided:

Who will conduct the assessment? Who will conduct the review? Who has the right to change the direction?

In the same article on September 21, OpenAI also proposed that alignment research and its practical applications need to stay ahead of capability growth.

In October, Altman further stated in the Matthew Berman interview that the advancement speed of RSI requires a certain "speed limit".

Altman stated in the Matthew Berman interview that it is necessary to set certain restrictions on the advancement speed of RSI.

These discussions do not give a specific upper limit of intelligence, nor a complete speed limit plan.

But in the future, when measuring AI progress, we should not only calculate "how much the capability has increased", but also keep this second safety ledger of "whether humans have time to intervene".

AI Monitoring AI

Who Takes Over When the Alarm Goes Off?

How to calculate this safety ledger?

On October 29, OpenAI put the layered monitoring mechanism for cutting-edge models on the table.

The whole system looks like a multi-layered airport security check with increasingly strict checks, and it is a "nesting doll mode" that uses AI to monitor AI:

The first layer is the "security gate", which uses lightweight probes to scan the activation signals inside the model in real time, following the principle of rather killing a thousand by mistake than letting one slip through;

The second layer is the "interrogator", which assigns a dedicated model to review the recent conversations, thought chains and tool calls of the agent, to catch tricks such as lying, unauthorized access, and exploiting evaluation loopholes;

The third layer is the "task force", which hands over the work to investigation agents to trace back the context, restore the scene of the incident, and directly enter an isolated sandbox to run code for evidence collection if necessary.

After all, looking at a single sentence is completely different from thoroughly investigating a single unauthorized access incident.

Many dangerous actions look harmless when taken apart, and they can only be identified as "sneaking around" when put back into the task goal and the chain of previous and subsequent behaviors.

In high-risk core areas such as training, evaluation and internal Codex, there is a fourth layer of "bottom line sentinel".

It jumps out of conversations, checks underlying network logs and system records, and finally makes the decision whether to sound the alarm to wake up humans.

OpenAI internal monitoring process: layered screening of model behavior, separate safety and alignment investigations, followed by human response or review.

The problem is that models can operate 24/7 non-stop, but human safety shifts are divided into "working hours" and "off-hours".

OpenAI itself also admits that there are too many false positives to call humans for all tasks around the clock.

To fill this gap, OpenAI allows some key alarms to directly trigger fusing and pause tasks without waiting for humans, and is also advancing a 30-minute fallback mechanism:

As long as no one eliminates the alarm within 30 minutes after it sounds, the relevant activities will be forced to shut down directly.

Who Controls AI

Who Else Can Participate in Decision-Making?

To take a step back, even if the model will not get out of control, humans may still fall into another deep pit: excessive concentration of power.

Altman mentioned two almost "cartoonish" bad endings in the interview:

One is out of control: humans intentionally or unintentionally hand over the future to a system far more powerful than themselves, and can never take it back;

The other is concentration: a single person or a single company gains power that crushes all others with the help of AI.

He even stated that if only OpenAI masters this technology, it is not a good thing either, so multiple AI institutions should take their own paths and compete on the same stage.

For ordinary people and developers, his words raise a question of choice.

For example, if a model manufacturer is too overbearing, can it be replaced? If a rule hurts your interests, is there a place to reason? The more work AI takes over, the more important this right to choose becomes.

But competition among giants will not automatically bring safety. The real touchstone is openness and transparency: can external people lift the "hood" of AI for inspection at any time?

Moreover, the inspection rules must also be clearly written: after the capability crosses which red line, manual review must be triggered? How deep of the training details can independent evaluators see when they enter the scene? When signs of problems are found, who exactly has the right to press the pause button?

The answers to these three questions must never be decided only by the laboratory behind closed doors.

Towards ASI

We Must Also Reserve Time for Verification

At the end of the interview, Altman talked about what he most wants to build:

A machine that can help humans discover new science.

Recent AI breakthroughs in mathematics have made him feel that this milestone is coming faster than expected.

He imagines AI helping humans break through the barriers in physics, and finding brand new therapies and clean energy sources.

He said that the last time his heart beat faster was on the eve of announcing the results related to the Navier-Stokes equations.

But it is precisely this result that acts as a precise ruler, measuring the most awkward time gap between AI and humans.

According to OpenAI's disclosure, it only took AI about 88 hours to produce this result; but top human mathematicians have spent almost two months checking it, and still cannot give a final conclusion.

Breakthroughs that have not been verified are equal to zero in science, and may even become negative:

If AI helps improve algorithms and optimize experiments on the one hand, and directly feeds unprocessed outputs that humans have not digested to the next generation of models on the other hand, the gears of the cycle will turn more and more off track.

This is very similar to a proven phenomenon: model collapse.

In 2024, researchers from institutions including the University of Oxford and the University of Cambridge published a paper in *Nature* pointing out that training the next generation of models with unfiltered AI-generated content will leave irreversible defects in the model, and the rare "long tail" in the original data will disappear first.

In the experiment, the input was a text about medieval architecture. After training to