Google DeepMind's Dream-RSI Cuts Agent Iterations by Up to 2.43x by Replaying Past Attempts

Google DeepMind has built a training method that lets AI agents improve by mentally replaying searches they have already run. The technique, called Dream-RSI, reuses the record of an earlier attempt to evaluate alternative strategies without paying for fresh rollouts each time. In the team's tests, agents using it matched or outperformed established baselines while shrinking the number of iterations needed by as much as 2.43 times.

The mechanism targets a familiar cost problem in agent research. Getting a language model to solve a task often means generating many candidate paths, scoring them, and retrying, and each retry burns compute. Dream-RSI instead treats a completed search as a kind of trace that can be re-examined, letting the agent ask what would have happened under a different decision at an earlier step. Because the underlying model is frozen, the method changes only the policy that governs how the agent searches, not the weights it was trained with.

That separation matters for how the work could be deployed. Teams that already run agents against fixed models, from coding assistants to retrieval pipelines, could in principle bolt on a revised search policy without retraining a base model or re-collecting data. The reported gains come from better decision-making about where to look, which branch to expand, and when to stop, rather than from more capable underlying reasoning. DeepMind has not published a full breakdown of the benchmark suite used, so the breadth of the improvement across task types remains unclear.

The result lands in a busy stretch of agent research, where labs are competing on how well systems plan over multiple steps rather than on single-shot answers. Reusing past experience is an old idea in reinforcement learning, but applying it to the traces produced by modern search-based agents is newer, and the compute savings are the headline. If the iteration reduction holds outside the tested settings, it would lower the cost of running agents that iteratively query tools, browse, or write and test code.

Several questions are still open. The summary does not say how Dream-RSI performs when the environment changes between runs, which is the normal case for live tools and websites, or how much memory the stored traces require at scale. It is also not clear whether the technique composes with agents that update their own model weights, since the current design assumes a frozen model. Watch for follow-up work measuring gains on long-horizon tasks and for any sign that DeepMind plans to fold the method into its agent products.