Two agents read the same document under different instructions. Their cached representations are not necessarily interchangeable, even though the source text is identical.
LatentRemedy's October 13 paper studies severe degradation from non-prefix cache reuse under conflicting tasks. Its remedy adds a small learned set of task-conditioned latent tokens after the reused block, trained through full-prefill distillation. Read the research paper.
Identical text does not imply identical cached meaning
Multi-agent systems often share a document while assigning different tasks to the models reading it. Reusing a cached representation seems like a natural efficiency: why process the same words again? The complication is that a cache is produced under a preceding context. Instructions before a shared block can shape its internal representations. A receiver asking a different question may inherit computation organised around the donor's task rather than its own.
LatentRemedy studies non-prefix cache reuse under conflicting instructions. This is a harder case than sharing a common prefix, where the computational history is aligned by construction. It tests whether a small intervention can make reused state more useful without recomputing the shared block.
A few latent tokens provide task-specific adaptation
The method maps the last hidden state of the target agent's prefix through a multilayer perceptron. That produces a small set of task-conditioned latent tokens placed after the reused KV block. The tokens interact with the shared context and are trained through full-prefill distillation to guide predictions toward the receiver's task. The intervention is therefore conditioned on what the receiver is trying to do, not simply on numerical differences between two caches.
This design reflects the authors' diagnosis: contaminated representations are only part of the problem. The receiver also has difficulty interpreting reused context under its own instructions. A repair aimed solely at copying or offsetting stored values can miss that functional mismatch.
The failure can be much larger than the saving suggests
The paper reports performance degradation of up to ninety percent from direct non-prefix reuse in conflicting-task settings. Existing selective-recomputation and offset-calibration remedies recover only part of the loss in the authors' comparisons. LatentRemedy consistently improves over the tested reuse baselines while keeping time to first token close to direct reuse as cache size grows.
These findings establish a problem and a proposed repair within the evaluated configurations. They do not show that any cache can be shared across arbitrary models or instruction sets. The learned adapter, compatible representations and distillation setup are part of the method. Their preparation and maintenance costs belong in an operational comparison.
One learned token helps, but the tested universe is limited
The Qwen3-4B ablation finds useful recovery even with one latent token. Increasing the token count does not produce a simple monotonic improvement: eight tokens are best in one non-reasoning comparison, while larger counts do not consistently help. The result supports a compact task cue rather than the idea that a longer adapter must always repair more of the mismatch.
The stated scope is Qwen3 at three scales. Latency is measured on one RTX 5090 with shared blocks up to 4,096 tokens. Other families, longer blocks and broader hardware remain untested. Those boundaries matter when translating an attractive first-token result into a general-purpose multi-agent cache-sharing service.
Shared infrastructure needs a semantic contract
A convincing deployment test should include conflicting tasks, irrelevant donor instructions and cases where fresh full prefill produces a clearly different answer. Similar tasks can make reuse look safe while avoiding the very disagreement that exposes its weakness. Exact full prefill provides a reference for what the receiver would have computed from its own prefix and the actual document.
Context infrastructure cannot treat cached computation as a neutral copy of text. It carries a history of interpretation. Efficient collaboration requires either preserving the conditions that make that history valid or adapting it explicitly. LatentRemedy offers one adaptation mechanism, while reminding developers that saving prefill changes more than the amount of work performed.
References
Questions, news tips or corrections? Contact [email protected].
