Google and DeepMind Propose Dream-RSI: Agents Improve Themselves by ‘Dreaming’ Over Past Searches

Google, together with Google DeepMind, the University of Maryland, and the University of Virginia, has published a paper proposing Dream-RSI, a framework that offers a new route to recursive self-improvement. Instead of modifying a model’s weights, Dream-RSI records an agent’s exploration steps as a search tree, then lets the agent “dream” by replaying that history to refine its exploration strategy.

The design separates execution from decision-making. A discovery agent, powered by either Gemini 3.1 Pro or Gemini 3.7 Flash, generates candidate code, while a lightweight orchestration layer contains the exploration strategy that decides which nodes to continue from and when to stop. Every real exploration run is stored in a historical tree; when a new strategy needs to be evaluated, the system only replays recorded nodes instead of running live code or calling the model, drastically cutting evaluation costs.

The team tested Dream-RSI in three areas. In algorithm engineering, Dream-RSI solved a Lasso regularization path task with 317 model calls, matching a fixed-strategy baseline that needed 550 calls and a SimpleTES baseline that generated 51,200 candidates. In circle packing, it matched the strongest record from Google’s AlphaEvolveV2. For GPU kernel optimization, Dream-RSI needed 2.43x fewer generations to reach the same performance as fixed exploration. The paper also found that adding human-written high-level guidance made agents perform worse, suggesting that code-level strategy refinement through “dreaming” preserves exploration diversity better than abstract experience summaries.

The paper is available on arXiv and a project page.