Google’s Dream-RSI: Agents That Get Better by Dreaming Over Their Own History
Dream-RSI turns an agent’s old work into a practice world. It tests new exploration strategies in that dream, then ships only the winner back to real work.
A Google and Google DeepMind team posted Dream-RSI on 14 September 2026 (arXiv:2609.14858), with a project site and open code. The paper’s line is simple: an agent must dream to recursively self-improve, and history is the world it dreams in.
This is not a claim that the model rewrites its own weights into AGI. It is a method for improving the exploration policy: the code that decides where to search next, what to try in parallel, and when to stop.
The problem it attacks
Long discovery jobs (better algorithms, math setups, GPU kernels) burn thousands of agent calls. Fixed search rules do not learn from dead ends. Trying new search rules online is costly, because you only learn if a rule worked after a full expensive run.
The trick: history as a free simulator
Every finished run already leaves a tree of attempts and outcomes. Dream-RSI treats that tree as an exact replay simulator for the ground already covered. No learned world model. No guessing. If a node was run once, replaying another policy over it costs zero new executions.
The loop has three steps:
- Explore online with the current policy and log the tree.
- Turn those trees into a growing replay pool.
- “Dream”: rewrite candidate exploration policies and score them by replaying history. Keep the best one. Because the old policy is always a candidate, the new one should not score worse.
Then deploy the winner online again. Each lap adds more history, so later dreams see a bigger world.
What they report
They test algorithm engineering, math optimization, and GPU kernel work. Against a fixed-exploration baseline with the same agent and budget, Dream-RSI often matches or beats quality at lower cost. Project figures include about 1.7× fewer discovery-agent calls vs fixed exploration, up to roughly 162× fewer vs a heavy baseline (SimpleTES) on one Lasso task, and about 2.4× fewer generations on a VGG16 kernel task at similar performance. Those are the authors’ reported numbers. Treat them as paper results, not an independent bake-off.
A notable finding: stuffing high-level “semantic tips” from past runs into the prompt often did worse than replay. Strong verbal hints can shrink search too early.
Why it matters
Recursive self-improvement sounds like sci-fi. Dream-RSI makes a narrower, useful claim: agents can improve how they explore by practicing on their own logs. That cuts cost and waste. It also raises a quiet safety point. Systems that rewrite their own search code need clear audit trails for what changed and why. Dreaming over recorded trees is easier to inspect than opaque online meta-learning, but it is still self-modification of behavior.
For builders, the practical lesson is blunt. Do not throw away discovery history. It may be the cheapest lab you have.
Sources
- Zheng et al., Dream-RSI: Recursive Self-Improvement through Evolving Worlds, arXiv:2609.14858, 14 September 2026. https://arxiv.org/abs/2609.14858
- Project site. https://www.dream-rsi.com/
- Code. https://github.com/zhengkid/Dream-RSI
Source: Zheng et al., Dream-RSI: Recursive Self-Improvement through Evolving Worlds