Shocking! Has Google cracked RSI? AI learns to evolve infinitely through "dreaming"

Google may have cracked RSI!

Just now, Google, together with Google DeepMind, the University of Maryland and the University of Virginia, published a high-impact paper.

This time, they demonstrated a completely different RSI path: AI self-evolution can even be achieved by "dreaming"?

This time, Google makes the agent store every step it takes as a tree, and then "dream" inside this tree.

It practices and develops a better exploration strategy in the dream, then returns to the real world to keep working.

Paper link: https://arxiv.org/abs/2609.14858

The performance of Dream-RSI can be described as quite stunning:

For algorithm optimization tasks, mainstream evolutionary search systems need to run 51200 generations, while Dream-RSI only makes 317 calls to reach the same level.

For the circle packing problem, it matches the strongest record of Google's own AlphaEvolveV2.

For GPU kernel optimization, to achieve the same performance, the number of generation calls is reduced by 2.43 times.

The underlying weights are not modified at all, and both Gemini 3.1 Pro and 3.7 Flash are fully compatible with this framework.

In addition, there is a more interesting conclusion: giving AI directional guidance manually leads to worse results than no guidance at all.

As soon as the paper was uploaded to arXiv, it immediately went viral on X. Netizen Mark Kretschmann shouted: "Google may have just cracked recursive self-improvement"!

The first author of the paper, Tong Zheng, is a second-year PhD student at the University of Maryland, who only joined Google as a Student Researcher this summer. He is backed by a 17-member author team, including DeepMind researchers and endowed chair professors.

To achieve recursive self-improvement, agents must learn to dream. And history is exactly the world where they dream.

Tong Zheng left this sentence on Hugging Face, which is also the core insight of the entire paper.

History is the world where it dreams

What Dream-RSI solves is the most bottlenecked link in the entire evolutionary search field.

Systems represented by AlphaEvolve have long proved that letting coding agents run evolutionary search can discover algorithms that humans have never found. But they have a fatal flaw: all their exploration strategies are hard-coded manually.

How much computing power to allocate to each branch, which branch should be pruned as early as possible, and when to enable parallel execution, all are pre-determined by engineers. Once the search space expands, the fixed strategy will waste computing power in dead ends, and it will not learn lessons from past experiences.

Then can we let the strategy evolve on its own? Theoretically it is feasible, but the problem lies in the evaluation cost.

To evaluate a piece of candidate code, you only need to run it once to know its quality. But to evaluate a set of exploration strategies, you need to let the agent run hundreds or thousands of rounds from the very beginning, check the quality of the final output, and then give a score to the strategy. Every time you modify a new version of the strategy, you have to re-run the entire discovery process, and it takes several days to get a single score, which no one can afford.

The solution of Dream-RSI for this problem is to store all the processes that have been executed, and replay the old processes directly instead of re-running everything when evaluating new strategies.

Specifically, it splits "executing tasks" and "deciding how to execute tasks" into two separate layers.

- The bottom layer is the execution "Discovery Agent", powered by Gemini 3.1 Pro or 3.7 Flash. Every time it receives a workspace, it reads the context, modifies a new version of candidate code, and submits it to the evaluator for scoring.

- The top layer is a lightweight orchestration layer, which contains the "exploration strategy" (a piece of Python code) that decides which node to continue from in each round, how many parallel attempts to start at one time, and when to stop loss.

The underlying model remains unchanged, and the only object that undergoes self-evolution is this piece of strategy code.

The result of every real exploration will be stored in a "discovery tree". The root node is the initial workspace, and each child node records the complete scene of an attempt, including the file system snapshot, generated output, evaluation diagnosis and score.

This tree is exactly the simulator.

The only type of decision the strategy needs to make is: which nodes to continue from, and how many attempts to start at one time. And what result you will get when you go down from each node has already been recorded in the tree.

So when the strategy needs to be iterated, you don't need to run code in the real environment anymore, you can directly walk through the history again on the fully grown tree ——

If you want to test "taking two more steps on this road", you can read the corresponding records in the tree; if you want to test "giving up that road earlier", you can read fewer nodes.

The entire evaluation process does not call the model even once, does not run a single line of code, and only reads historical data.

The paper calls this mechanism "dreaming". Just like a chess player reviewing a game, there is no need to reset the chessboard and play the game again, reviewing the chess record is enough.

The operation closed loop of Dream-RSI

Based on this insight, Google built the Dream-RSI framework. Its self-improvement loop is mainly divided into three stages:

1. Real exploration: the latest exploration strategy at present

guides the Agent to run in the real environment (for example, call LLM, compile and execute code), and records all paths, decisions and execution feedback generated by the exploration, as a new structured exploration tree

to append to the history database

.

2. Build the simulator: convert the updated history database

into a reusable "replay simulator pool".

3. Dream-based strategy improvement: a dedicated LLM strategy development Agent starts to "dream" crazily in this simulator. It continuously modifies and rewrites the code of the exploration strategy, generates thousands of candidate strategies, and quickly replays them on the replay simulator.

The scoring of the dream consists of three parts: first, the optimal score found in this round of dream, second, subtract the cost corresponding to the number of attempts, third, the parallel reward, which encourages the strategy to run multiple attempts in batches in one round.

A high-score strategy means it achieves better results with faster speed and lower cost.

The one responsible for modifying the strategy is another "strategy development agent".

It will focus on three things: the trajectory and score of the current strategy in the dream, the feedback of previous versions of modifications, and the entire discovery tree. Then it will directly rewrite the strategy code after reviewing all these contents.

Several versions are modified in one round, each version is sent back to the dream for scoring, and the version with the highest score becomes the official strategy for the next round.

This means that the lower limit of the new strategy is "staying at the current level", and the upper limit is "infinite evolution".

The RSI flywheel is thus closed.

Why can this strategy never get worse?

In the offline improvement stage, the strategy development Agent will go through multiple rounds of code iteration. Since the current strategy

itself is included in the candidate pool, the new rewritten strategy will only win and be deployed when its comprehensive replay score on the simulator (which balances exploration quality, computing power cost and parallel efficiency) is higher than the current strategy

.

Therefore, mathematically, the performance of the new strategy deployed by Dream-RSI will never be weaker than the previous generation. As the new strategy is redeployed to the real world for exploration, it will bring back a deeper and wider exploration tree, so that the simulator will also evolve accordingly.

The agent and the "world" in its eyes grow together in this interlocking gear system.

After the strategy is modified and launched to discover new algorithms, the historical data increases, the dream becomes more accurate, the dream becomes more accurate, the strategy continues to evolve. Branches that do not exist in the tree cannot be reached in the dream, so every round of real exploration expands the scope of the dream for the next round

The first version of the strategy is very simple, it only starts a bunch of independent workspaces and runs them separately. In subsequent rounds, the strategy learns to be cost-effective.

In the ConvDiv kernel experiment, when the performance is on the rise, the strategy actively cuts the number of attempts per round from 110 to 50; once the performance stagnates, it increases the exploration intensity to the maximum, and the score immediately rises further.

No one taught it to do this, it "realized" this rule from its own history.