Switching to another model normally means processing the context again. Translating the first model's cache could avoid that cost, but matching stored numbers is not enough.
CacheLink's October 11 paper finds that similar key-value reconstruction errors can produce different attention and answers. Its correction objective targets the receiving model's behaviour, improving evaluated handoffs between independently trained model pairs. Read the research paper.
Switching models should not require forgetting the context
An assistant may begin with a cheaper model and escalate to a stronger one when a task becomes difficult. The receiving model normally processes the context again, because the first model's KV cache is not automatically compatible with its own internal states. At long context lengths, that handoff can be expensive enough to discourage useful model switching. Translating a cache offers a possible shortcut, but the translation needs a meaningful definition of success.
CacheLink examines zero-prefill cross-model handoff. Its central finding is that conventional reconstruction fidelity is a poor stand-in for whether the receiver will make good use of the transported state. Similar stored numbers need not produce similar attention or answers.
The receiver's computation defines the target
Earlier affine transport methods show that receiver keys and values can be substantially reconstructed from a sender cache. CacheLink's analysis finds that similar reconstruction errors can nevertheless yield different attention patterns, predictive distributions and downstream accuracy. The importance of an error depends on how the receiver's queries interact with it, not only on its average squared magnitude.
The proposed method learns a coupled correction for keys and values using receiver behaviour directly. This treats the pair as part of a functional attention computation. A correction that barely changes a tensor-distance score can still matter if it changes which evidence the receiver uses for its next prediction.
Small numerical changes can have large behavioural effects
Across four independently trained sender-receiver pairs and seven evaluation panels, the authors report that CacheLink improves the equal-weight mean over all compared methods for every pair. In matched diagnostics, key-value squared error changes by less than 0.4 percent while attention and next-token behaviour change substantially. That observation is the clearest reason not to use reconstruction error alone as the deployment criterion.
At a thirty-two-thousand-token context, the reported handoff latency is up to 7.27 times lower than re-prefilling the receiver. This is a handoff result in the evaluated configurations. It does not imply unrestricted interchangeability among models or an equivalent acceleration of the complete generation that follows.
The protocol changes transport while holding the task fixed
The evaluated pairs span Qwen3, Ministral reasoning models and Yi, with smaller-to-larger handoffs within each pair. Every method constructs a complete receiver cache and performs no receiver prefill on the reusable prefix. Within a pair and evaluation panel, tokenised inputs, prompts, decoding rules and scorers remain identical.
Keys are translated after undoing the source rotary position transformation and then receive the target transformation before injection; values remain in raw cache space. These details matter because compatibility includes positional conventions as well as tensor shape. The seven panels also contain both short multiple-choice tasks and long-prefix tasks, so the summary result should be read alongside individual panels rather than treated as one homogeneous accuracy measure.
Escalation needs to preserve the difficult evidence
The important deployment cases are often precisely those where a cheap model was insufficient. A translated cache should therefore be tested on rare facts, conflicting instructions and multi-step dependencies, not only on common context that both models handle easily. Fresh receiver prefill remains the reference for understanding which differences the shortcut introduces.
Learning and maintaining transport for supported model pairs also belongs in the cost account. Cache transport could make escalation more economical by sparing the stronger model a fresh pass through the same context. CacheLink sharpens the criterion for that opportunity: a handoff is successful when the receiver's computation remains useful, not when the transported tensors merely resemble what a numerical reconstruction metric expects.
References
Questions, news tips or corrections? Contact [email protected].
