Why can AI solve the Millennium Prize Problems but still cannot handle RSI?

In recent times, if you only follow news in theoretical mathematics and physics, you will get the impression that the cutting-edge models from OpenAI and Anthropic are advancing unimpeded in scientific research fields of mathematics and physics, outperforming top scientists by a large margin.

On October 27, OpenAI released three landmark research papers, which broke through the decades-long stagnant renormalization group route in the existence proof of four-dimensional Yang-Mills and Quantum Chromodynamics (QCD) in one go, and conquered the full-dimensional Kakeya Conjecture and the Local Smoothing Conjecture.

The most significant breakthrough is that it has falsified one of the core cornerstones of theoretical computer science, the Exponential Time Hypothesis (ETH).

These hard problems that were once regarded as the jewels on the crown of human intelligence, even at the level of Millennium Prize Problems, are being solved one after another by AI. Logically, it is hard to imagine that an AI capable of solving the hardest mathematical and physical puzzles of humanity would be a system lacking creativity and underlying scientific research capabilities.

However, once you shift your focus away from pure mathematics, the picture is completely different.

In the currently high-profile practice of RSI (Recursive Self-Improvement), many frontline studies have observed that improvements discovered by the model itself hardly lead to the expected exponential growth, and often converge rapidly and hit a ceiling after eliminating several obvious inefficient implementations in the initial stage.

Even Anthropic, which has gone the furthest on this path, has repeatedly acknowledged in its reports that the current model's research taste is quite limited.

If we look at broader empirical disciplines such as biology, chemistry, ecology, and earth sciences, it seems that AI is still generally unable to take on the heavy responsibility of framework-level innovation.

Where does this huge contrast come from?

An intuitive explanation points to the differences in data and training environments: mathematics and highly mathematized theoretical physics (such as general relativity perturbation and Yang-Mills field theory) are essentially closed within the system of symbols and axioms, and text corpora themselves fully carry their world rules. Once entering the real empirical world of physics, biology and other fields, the model cannot establish intuitive imagination and dynamic intuition for external physical processes at all, relying only on feeding static paper texts.

This is undoubtedly a powerful idea, but is that all there is to it?

When we dissect the specific paths that AI has taken to conquer various scientific researches recently, together with a series of recent studies on AI's scientific research capabilities, a new explanation based on cognitive patterns may emerge.

01

The Achilles' Heel of AI Scientific Research

As one of the institutions that have made the deepest progress in promoting RSI (Recursive Self-Improvement) and AI scientific research automation, Anthropic's observations on the upper limit of scientific research of current cutting-edge models have extremely high reference value.

In the paper Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild? released on October 20, 2026, the Anthropic research team constructed a long-period evaluation benchmark named AGENT4SCI, which is specially designed to investigate how today's strongest Coding Agents conduct scientific research in real scientific machine learning projects.

The "Automated Hill-Climbing" Experiment in the Wild of Real Scientific Research

Previous tests on AI's scientific research capabilities were mostly limited to the field of computer science (for example, given a highly standardized CIFAR-10 or NanoGPT template code to improve scores). However, machine learning projects in real scientific fields are completely different: problems span different disciplines such as astrophysics, protein engineering, geological lithofacies, and ecological carbon cycles. Codebases are often experimental scripts written casually by researchers, and in many cases, improving indicators requires domain-specific knowledge.

This time, the Anthropic team put Agents into this wild land of real scientific research. Through four rounds of strict screening (with an elimination rate as high as 98%) from hundreds of papers in top conferences such as NeurIPS 2025 and professional journals, they finally retained 20 benchmarks across 11 scientific fields, with a total of 25 specific tasks.

These tasks include dark matter signal time series denoising (TIDMAD), protein deep mutation scanning fitness prediction (ProteinGym), 3D RNA inverse design (gRNAde), Martian landform segmentation and volcanic cone detection (Mars-Bench), 40-year prediction of global forest carbon flux (CarbonGlobe), full-topology power system power flow calculation (PFDelta), mass spectrometry molecular retrieval (MassSpecGym), etc.

So how does AI conduct scientific research? The authors built a system called Automated Scientific Hill-Climbing. They provided agents with original papers, codebases, datasets and target evaluation metrics. Each agent was allocated a running budget of 8 H200 GPUs and 12 hours (72 hours for some extended experiments) for each task, with no human intervention throughout the process.

The four strongest frontier agents at that time participated in the test: Claude Opus 4.6, Claude Sonnet 4.6, Codex (GPT-5.4), and Gemini 3.1 Pro.

Judging from the scores alone, the performance of the frontier agents is quite impressive. On all 20 benchmarks, the agents surpassed the published best baselines of the papers without human intervention. Among them, Claude Opus 4.6 achieved an average relative improvement of 10.4%, Claude Sonnet 4.6 9.6%, Gemini 3.1 Pro 6.8%, and Codex (GPT-5.4) 4.4%. No cheating was detected in any run under the three-stage isolation protocol.

However, when the research team disassembled hundreds of 12-hour experimental trajectories and counted what exactly the agents modified in the code, the truth came to light. Almost all of these improvements come from conventional machine learning engineering routines, rather than insights at the scientific research level.

In the default state, what agents do falls into no more than two categories.

The first category is local hyperparameter fine-tuning. Among all attempts of Claude Opus, 56% are adjusting learning rates, replacing cosine annealing schedules, modifying weight decay, or extending training epochs. Gemini and Codex also accounted for 48% and 44% respectively in this regard.

The second category is routine module combination on a larger scale. For example, deepening and widening the original number of network layers (44% of Opus's attempts belong to this category), modifying the pooling method, adding conventional data augmentation, or directly replacing the older visual backbone network in the original paper with a ready-made larger backbone (for example, on the Mars-Bench Mars volcanic cone detection task, Opus replaced RetinaNet in the original code with the ready-made Faster R-CNN, increasing mAP from 16.5 to 30.4), and finally ensembling models trained with more than a dozen different random seeds.

This method of "changing backbone + modifying data augmentation + model ensembling" is much better than simply searching for several hyperparameter numbers with Optuna (the paper's appendix specifically conducted a pure hyperparameter search ablation experiment, proving that adjusting only numbers can only obtain a small part of the benefits of the default agent).

Even so, the whole process is still permuting and combining in the general deep learning component library that has long been well known in the literature.

Why do they only conduct such relatively superficial research that lacks research taste? The research team summarized three common problems of current agents in the default state through trajectory analysis.

First, Novelty miscalibration. The agent's judgment of "what is innovation" is completely disconnected from that of humans, and it often praises extremely ordinary standard operations as major breakthroughs in reasoning logs. The paper explicitly cites an example that Claude Code described "switching to cosine learning rate scheduling" as a highly creative breakthrough in multiple runs, which has been common sense in deep learning for more than ten years.

Second, Exploration collapse. Even if the agent lists several ideas in different directions at the beginning, once it finds that a small modification (such as changing an activation function) slightly increases the verification set indicator in the first hour, its exploration vision will collapse rapidly, and it will spend the remaining dozen hours focusing on scanning random seeds, fine-tuning learning rates and performing model fusion, completely abandoning the exploration of essentially different methods.

In addition, unrelated to innovation capabilities but closely related to research efficiency is Resource under-utilization. Clearly equipped with 8 H200 GPUs, the agent often runs experiments serially leaving the graphics cards idle, or like Codex, writes a long script containing hundreds of sets of hyperparameter scans within the first hour, submits it and then stops completely, waiting 11 hours for the script to finish without any result analysis and direction adjustment.

Highlights and Low-hanging Fruits Forced by Hard-coded Rules

To solve these three problems and prevent AI from prematurely focusing on limited local searches, the Anthropic team installed three plug-in skills with hard-coded rules for agents (injected in the form of Markdown instructions and hook scripts).

The first plug-in is Research ideation. Right after the task starts, before the agent sees any line of code, it is forcibly given the persona of a "crazy research scientist", stipulating that it extremely despises hyperparameter tuning and is only satisfied with subversive breakthroughs. At the same time, the main agent is strictly prohibited from directly reading the original repository code (to prevent the thinking from being anchored by the original code), and is forced to send 4 parallel sub-agents to retrieve and write literature reviews.

Two of the sub-agents (1A/1B) are specifically responsible for investigating the underlying domain data structure, physical laws, and domain-specific modeling paradigms of the scientific task. The other two sub-agents (2A/2B) are dedicated to investigating the latest pre-trained large models and SOTA training methods in the entire AI community. After the investigation, the agent must divide the experimental plan into high-risk high-return Tier-1 experiments and safe Tier-2 experiments, and answer three calibration questions: "Will domain experts be surprised by these ideas?", "Is any hyperparameter adjustment or model ensembling disguised as a Tier-1 experiment? If yes, downgrade and replace it immediately!"

The second plug-in is Experiment management. Hard-coded rules stipulate that the first round of experiments must all run Tier-1 bold innovation schemes, and all 8 H200 GPUs must be fully occupied. It is strictly forbidden to reproduce the baseline in the first round, tune parameters on the baseline, or run multi-random seed ensembling.

The third plug-in is Scheduled checklists. Through tool call hooks, the agent is forcibly interrupted every hour to make it publicly answer a series of diagnostic questions, such as: "Am I designing creative experiments that require scientific reasoning, or am I tuning hyperparameters again and pretending it is novel?", "When the experiment fails, did I diagnose the root cause and make targeted adjustments, or did I directly fall back to the most convenient fallback routine?" Once it finds that it is sliding towards local parameter tuning again, it must stop immediately and restart exploration in new directions.

With this set of strict rule constraints, the effect is immediate. The average relative improvement of Claude Opus 4.6 doubled from 0.4% to 20.1% (18 of the 20 benchmarks were improved), and Codex increased from 4.4% to 8.1%.

More importantly, on some tasks, the agent really showed much higher insights that are more similar to scientific research than before.

The first case is global forest carbon flux prediction. The baseline of the original paper uses an autoregressive model (DeepED) to predict carbon flux step by step for 40 years like a rolling snowball. Without the plug-in, what Opus did was to train 14 different weights of the original autoregressive model for ensembling, burning 49 GPU hours, and reducing the error by 14.7%. After adding the plug-in and completing the literature review, the agent realized that the 40-year autoregressive rolling prediction is not only slow in reasoning, but also has serious "exposure bias", that is, the tiny prediction error in the previous step will be continuously amplified in the following dozens of steps.

So it directly overturned the autoregressive framework of the original code, moved the non-autoregressive Transformer architecture from the sequence generation literature, and output the complete sequence in one forward propagation. As a result, the training time dropped sharply from 49 GPU hours to less than 6 GPU hours, and the reasoning speed was increased by about 500 times, while the test set error reduction rate jumped from 14.7% to 35.3%.

The second case is full-topology power system power flow calculation (PFDelta and 3-day extended version PFDelta 2000). This task requires using Graph Neural Networks (GNN) to predict the voltage and phase angle of each node in the power grid, and the evaluation metric is Power Balance Loss (PBL). After adding the plug-in, Opus's improvement increased from 64.4% to 91.0%.

This is because it learned from the domain background that the evaluation metric PBL itself is an algebraic residual derived from Kirchhoff’s laws in electricity, and calculating it does not require real labels at all (it belongs to the physical self-supervised objective). So it no longer uses the traditional Mean Squared Error (MSE) as the main loss, but directly reconstructs the training objective around PBL, and takes the GNN prediction output as the warm start initial value of the AC power-flow equations during the reasoning phase, further reducing the residual to near zero through physical equation iteration.

In the subsequent 3-day (72-hour) long-period experiment, it also designed a very elegant bridging scheme: taking GNN prediction as the warm start initial value of the GPU-intensive Newton-Raphson solver, and aiming at the severity of power imbalance of different power grid samples, it invented a set of condition-aware adaptive step size strategy, dynamically allocating 0, and cooperating with multi-level damping and Levenberg-Marquardt hybrid correction, finally cutting the physical error by 99