After the trillion-dollar AI bet, Ilya Sutskever: How far off can the results be if you just throw in computing power and are willing to do research?
A trillion-dollar bet has been placed.
Gartner predicts that global AI spending will approach $1.5 trillion in 2025 alone and exceed $2 trillion in 2026. Jensen Huang, the CEO of NVIDIA, believes that the investment in AI infrastructure this decade could reach $3 trillion to $4 trillion, calling it a new industrial revolution.
Everyone is scrambling for GPUs, building data centers, and expanding power grids. It seems there's only one question left: How much more computing power can we pile on?
Ilya Sutskever, the former chief scientist of OpenAI and the founder of Safe Superintelligence Inc. (SSI), gave a completely different answer in the latest episode of The Dwarkesh Podcast on November 25, 2025:
We're moving from the age of scaling to the age of research.
The person who said this is precisely one of the first to push the computing power hypothesis to the extreme. After leaving OpenAI in 2024, he founded SSI, a company that focuses on one thing: safe superintelligence.
In less than a year, SSI completed a $3 billion financing round at a valuation of $32 billion.
In this 90-minute interview, he made three core judgments:
The transferability of today's large models is far inferior to that of humans.
Continuing to pour money into buying parameters, data, and computing power is resulting in a rapid decline in marginal returns.
What truly sets the industry apart is not who has more resources, but who understands how to conduct research.
As the era of "piling on computing power" gives way to the era of "conducting research," the underlying logic of the AI industry is being rewritten.
Section 1 | The Era of Piling on Computing Power Is Almost Over
Ilya started by making a judgment:
We are moving from the era of scaling to the era of research.
The so-called era of scaling refers to the idea that as the three elements of parameters, computing power, and data are continuously amplified, the capabilities of the model will continue to increase. Leading laboratories such as OpenAI, Anthropic, and Google DeepMind have all used this approach, and it was effective for a while.
But Ilya believes that this era is reaching its peak.
"Scaling has become an industry consensus: buy more GPUs, build larger data centers, and train larger models. As long as the method remains the same, whoever acts faster and has a larger budget will take the lead."
The problem is that this approach leads not to innovation, but to a resource arms race.
Ilya himself is a firm proponent of the scaling approach. The GPT-2 and GPT-3 models he led were typical products of the scaling paradigm. But now he believes that simply piling on parameters has reached a bottleneck.
SSI is betting on a technological direction: The difference in future superintelligence lies not in who has more GPUs, but in who can find new training methods.
In the AI field, it's no longer about who invests more, but about who knows where to make a breakthrough.
He even said bluntly:
"Today's models have high evaluation scores, but the actual economic value they generate is limited. They seem to be very capable, but when you actually put them to work, you'll find problems."
You think they're highly capable, but it's just a good-looking benchmark score; you think the gap is small, but when you actually deploy them, you'll find problems everywhere.
The core reason for the diminishing marginal returns and the disconnect between capabilities and performance is simple: While computing power and parameters are still important, they are no longer the decisive factors for the model.
Next, let's see what Ilya means by the era of research.
Section 2 | Models Can Pass Exams, but Can't Do Real Work
Why do models look good on benchmarks but have so many problems in reality? Ilya's answer is that there's a problem with the model's generalization ability.
"On the one hand, it can write papers and solve math problems, but on the other hand, it can repeat a sentence twice."
This is not a single bug, but a systematic flaw: Just because a model can pass an exam doesn't mean it really understands. The problem lies not only in the model itself, but also in the people training it.
Ilya mentioned a phenomenon in the interview:
"The way we train models relies too much on evaluation benchmarks. Research teams design RL training environments specifically to boost their scores on the leaderboard."
Training resources are overly concentrated on a small number of tasks, such as competitive programming and standardized tests. The models are indeed getting stronger, but they're becoming more like test-taking machines, only capable of a few things.
He even said:
The real reward hackers are not the models, but the human researchers gaming the benchmarks.
He used two students as an analogy:
- Student A: Practices competitive programming for 10,000 hours, solves all the problems, and ranks among the top.
- Student B: Practices for only 100 hours, but has his own system for understanding problems.
"Who will perform better in a real-world career? Undoubtedly, it's the second student. Because he doesn't just memorize answers, but grasps the essence of the problems. Most of today's large models are like the first student."
Today's models lack not the ability, but the ability to judge what's worth learning.
Ilya doesn't deny the knowledge capabilities of large models: In math, language, and programming, they're even better than ordinary people. But they learn more slowly and are more prone to errors when dealing with new situations. Humans can intuitively judge whether they really understand, but models can't.
What he wants to explore is the problem with the training method:
- Pre-training: Put all the data in without selection, and the model will know a little about everything.
- Reinforcement learning (RL): Humans set tasks and rewards, and the model optimizes according to the goals, but it's prone to overfitting the rewards.
- Generalization ability: Can the model handle tasks it hasn't been trained for? Can it transfer existing knowledge? Can it correct itself?
The root cause of the problem that models can pass exams but can't apply their knowledge lies in the training process, which fails to teach them to think flexibly.
Therefore, the new breakthrough in AI is not about whose model is more capable, but about whose training method can truly teach the model to generalize and apply knowledge to new scenarios.
This requires not just adding a few more RL environments or doing more practice questions, but reconstructing the training strategy itself.
Section 3 | Why Current Training Methods Don't Work
In Section 2, we talked about the generalization problem. But why is it so difficult to solve?
Ilya's answer is that it's not a lack of resources, but a fundamental limitation of the training method itself. The dilemma of pre-training: It has seen a lot, but doesn't understand deeply.
This doesn't deny the value of pre-training.
Ilya clearly pointed out two advantages of pre-training:
The data is comprehensive and large in quantity, covering a wide range of behaviors.
There's no need for manual selection, and the training process is highly automated.
But he also pointed out the fundamental limitation of pre-training: It seems like the first 15 years of human experience accumulation, but humans learn much less and understand much deeper.
Humans don't make the same basic mistakes as models. Pre-training exposes the model to 10,000 programming cases, but it can't figure out when to use recursion or loops on its own. It's just copying, not really reasoning.
RL tries to teach the model goal-oriented behavior, but it brings new dilemmas: Research teams have to manually design tasks and define rewards. This leads to two problems:
First, the model only learns a few tasks, not how to learn.
Second, the model over-optimizes the reward function and loses its understanding of the essence of the task.
Ilya mentioned a crucial missing element: "Value function."
When humans learn, they have an intuitive judgment of how well they're doing, which allows them to correct themselves and transfer experience. But current RL methods can't give the model this ability.
Ilya used a scenario to summarize the limitations of current methods:
"You ask the model to fix a bug, and it says, 'You're absolutely right, I'll fix it.' After fixing it, it introduces another bug. When you point it out, it says, 'You're right again.' Then it brings back the first bug."
This isn't because the model isn't smart; it's because it has no judgment mechanism.
It doesn't know if it really understands or just guessed right, if a direction is worth pursuing, or how to evaluate its own reasoning process.
Both pre-training and RL are "offline learning": All learning is completed during the training phase, and then the model is deployed. This means the model can only perform well on known problems and is prone to unpredictable behavior in unknown scenarios.
More importantly, this paradigm can't teach the model the most crucial ability: To judge what's worth learning, when it's learned correctly, and how to transfer existing knowledge.
This is why simply increasing the scale of parameters, data, and computing power can't solve the fundamental problem.
You can make the model larger, but if the training method remains the same, it will always be just a sophisticated test-taking machine, not a real learner.
Section 4 | Ilya's New Answer: Enable Continuous Learning for Models
If the first three sections discussed the diminishing returns of the scaling logic, what Ilya really wanted to convey in this interview was a deeper signal of change:
AI safety isn't a problem to be considered only before product launch; it starts the moment you decide how to train the model.
The training method itself determines whether the model is safe and reliable in unknown scenarios.
Many people think that safety issues mean the model should follow rules or not lie.
But Ilya believes that the alignment problem is essentially a lack of generalization ability. When the model enters the real world, it can't understand which behaviors are allowed and which shouldn't be attempted.
It's not that the model is "bad" and acts maliciously; it's that it doesn't understand the context.
It's not that humans didn't set up the rewards properly; it's that the model hasn't learned to judge long-term impacts.
This turns the alignment problem into a more fundamental one: What exactly have you taught the model? How does it know it has learned? How does it reason about unknown tasks?
If the model just memorizes answers, it's a ticking time bomb; but if it understands the reasoning principles, it's more like a person who understands boundaries.
In the interview, Ilya clearly stated that he no longer believes in the one-time pre-training approach:
Humans are different from AGI. We rely on continuous learning.
A true intelligent agent doesn't complete learning during training, but continues to learn after deployment.
He used an analogy to explain: You can train a super-smart 15-year-old, but he doesn't know anything. True ability comes from how he enters society, receives feedback, and gradually acquires skills.
This isn't just an issue of ability; it's also a safety issue. This way, we can prevent the model from going out of control in unknown situations.
Ilya knows well that the scaling approach can't support continuous learning:
Pre-training is a fixed learning phase and can't adapt in real-time.
RL strongly depends on reward design and is prone to over-optimization.
The evaluation mechanism focuses on good results, regardless of whether the process is reasonable.
Therefore, he emphasizes that we need new methods to continuously calibrate the model's reasoning ability during the learning process. The real breakthrough isn't a larger model, but a model that can self-evaluate.
This isn't just a minor adjustment to the training strategy; it's a paradigm shift:
- From offline pre-training to online learning during deployment,
- From one-way optimization of goals to interactive multi-round feedback,
- From closed datasets to dynamic tasks in the open world,
- From alignment evaluation to the alignment process itself.
He also proposed that if we can design a structure that gives the model a mechanism similar to the human emotional center, it might be the path to building a trustworthy superintelligence.
Ilya gave an example:
"Why is Linux more secure today? It's not because we planned everything perfectly from the start, but because it's been deployed in the real world, used, attacked, and patched."
He believes that AGI must go through a similar process: Gradually deploy, receive real feedback, keep the structure controllable, and the mechanism transparent, rather than developing in isolation and finally releasing a black-box model.
That's why he decided to found SSI, which focuses on one thing: building a superintelligence that can continuously learn, align with humans, and be gradually deployed. It doesn't develop applications or tools; the product is the intelligence itself.
True safety isn't just a compliance slogan; it's a training philosophy.
From the first line of code and the first training sample, the future direction of the model is being determined.
This also means that the real gap in AI is shifting from resource scale to method innovation.
Conclusion | The Return of Research: A Technological Turning Point
In the 90-minute interview, Ilya's stance was clear:
The returns from scaling are diminishing, and research ability has become the key to differentiation.
Correspondingly, the evaluation system is also changing. The improvement of model capabilities no longer depends on simply increasing the scale of parameters. The pre-training phase alone can't solve the generalization problem of "learning by analogy." Continuous learning has become a necessary condition for ensuring safety. "Alignment" is no longer just a pre-launch inspection; it runs through the entire training process.
The logic of the scaling era, which involves calculating GPU costs, analyzing ROI, and chasing benchmark rankings, is becoming ineffective. The question isn't whether more computing power can make the model stronger, but whether this path is viable.
Safe superintelligence isn't just a concept in papers and consensus. It's the result of the synergy of technological paths, organizational structures, and business logics.
This turning point has arrived. It's still uncertain who will seize the opportunity.
📮 References:
https://www.youtube.com/watch?v=aR20FWCCjAs
https://x.com/dwarkesh_sp/status/1993371363026125147
https://www.gartner.com/en/newsroom/press-releases/2025-09-17-gartner-says-worldwide-ai-spending-will-total-1-point-5-trillion-in-2025
https://ssi.inc/
https://www.reuters.com/technology/artificial-intelligence/openai-co-founder-sutskevers-new-safety-focused-ai-startup-ssi-raises-1-billion-2024-09-04/
https://www.dwarkesh.com/
This article is from the WeChat official account "AI Deep Researcher," written by AI Deep Researcher and published by 36Kr with authorization.