Independent sampling asks a model to try again without necessarily using what the previous attempt revealed. For a verifiable problem, that can mean paying repeatedly for the same wrong answer.
LEAF's October 13 paper inserts problem-specific memory containing recent responses, rewards and feedback before new rollouts. Policy updates use self-distillation separately from exploration. The authors report faster exploration and held-out accuracy gains on their tested models. Read the research paper.
Why another attempt should know about the last one
Reinforcement learning with verifiable rewards has an unusually clear source of feedback: a proposed solution can be checked. Yet clear feedback does not guarantee economical exploration. Independent rollouts can repeatedly enter a dead end that an earlier attempt already exposed. Increasing the number of samples buys more chances, but it can also buy more duplication. LEAF treats that duplication as an information-management problem inside training, rather than assuming that a larger sampling budget is the only route to better answers.
The memory is specific to the problem being attempted. This is not a general store of user preferences or a retrieval database assembled from unrelated tasks. It is a working record of what the current search has just learned.
A short-lived record changes the search
Before each rollout, LEAF supplies the most recent response, its reward and available environmental feedback. A verifier call updates the record before the next attempt. The policy therefore generates under conditions different from those of an independent sampler: the next response can react to a known failure. The subsequent policy update is handled through self-distillation, separating the process that finds useful behaviour from the process that makes behaviour more likely in the model.
That separation makes the system interesting beyond its memory component. A useful training aid need not be a capability the deployed model must reproduce internally at every step. It can change which experiences become available to learn from.
The reported gain has two different meanings
Across four benchmarks and two open models, the authors report roughly fourfold faster exploration than a random-sampling baseline. They also report held-out accuracy improvements of 1.59 and 1.83 percentage points over their strongest baseline on the two models, using less compute than the RL baselines in their comparison. Those are separate claims. Faster discovery says that the training system finds rewarding responses more efficiently; held-out improvement addresses whether the resulting policy benefits beyond those particular attempts.
The distinction prevents an easy misreading of memory-assisted learning. A search procedure can look impressive because it carries unusually helpful context. The final checkpoint must earn its own evaluation after that context is gone.
The last failed answer carries information a verdict cannot
The exploration ablations distinguish the response from its feedback. Removing the compressed previous answer lowers performance on both tested models. Removing feedback also matters, but not equally across models: the reported effect is smaller for Olmo-3-7B-Instruct than for Qwen3-8B. The authors suggest that seeing the concrete attempt gives the model something to edit, whereas feedback alone can amount to learning that an unseen answer was wrong.
This is an important design detail for feedback-driven search. A failure label and a failed example are not substitutes. The example preserves the route taken and the assumptions embedded in it. The ablation therefore supports retaining enough of the attempt to make correction actionable, rather than reducing every unsuccessful rollout to a score.
What an engineering comparison should retain
A faithful comparison should retain the cost of failed verifications, context processing and the distillation stage. It should also distinguish a memory that prevents one exact answer from recurring from a memory that helps identify a better strategy. The former can improve sample efficiency without delivering much conceptual transfer. The latter would be more valuable on new problem families, but requires stronger evidence.
For a team training a coding or reasoning model, the immediately useful experiment is modest: record a failed attempt, pass its concrete feedback to the next rollout, and compare both discovery cost and the eventual memory-free policy. That tests the two proposed benefits without treating them as interchangeable.
Questions, news tips or corrections? Contact [email protected].
