A few-shot multimodal prompt can carry a large hidden serving bill. Each example image expands into visual tokens, and every new query processes those examples again. When the task remains stable across thousands of requests, repeatedly encoding the same lesson can become more expensive than learning a compact task-specific representation once.
STAVE investigates that alternative while leaving the backbone frozen. It compresses the task into two vectors added to existing input embeddings. The design is deliberately small, but its meaning is not simply “zero-shot.” The task has been learned from labels; the demonstrations have been removed from the inference path.
Answer tokens and context tokens get different adjustments
STAVE, introduced October 6, replaces demonstrations with two task-specific vectors added to existing embeddings. One affects answer-producing tokens; the other affects structural context tokens. The vectors are trained from labelled prompts, while the backbone remains frozen. Read the research paper.
One learned vector modifies answer-producing tokens, while the other modifies structural context-token groups. The two vectors are trained on labelled prompts with and without demonstrations. Rather than adding extra tokens or keys that compete for attention, the method changes the representation of tokens already present.
The distinction between readout and context is central. Adapting the place where an answer is produced is not necessarily the same as adapting how the rest of the prompt is interpreted. The paper supports its choices through a first-order loss analysis and a margin bound, alongside empirical evaluation.
The adaptation surface stays independent of model depth
Earlier demo-free methods can add task parameters at every decoder layer or search for task-specific insertion points. STAVE instead modifies existing input embeddings with two vectors. Its compactness therefore comes from a different intervention surface, not just from choosing fewer demonstrations.
That choice also changes what a human can inspect. A text prompt shows the examples and instructions being presented; embedding offsets do not expose their learned effect in readable form. This is not a reason to reject the method, but it changes the debugging interface. A useful operational record should retain the fitting examples and labels alongside the vectors, then test output behaviour when task definitions change. The parameter count is small enough to make storage inexpensive, yet the semantic commitment can still be substantial. Compact task state is an engineering advantage when its identity and intended scope remain tied to a reproducible adaptation process.
Small task state still requires a lifecycle
The authors evaluate six large multimodal models and five language models, reporting competitive or better results against compared methods with fewer task parameters. On eighteen text tasks, they report surpassing 15-shot prompting and earlier task-vector approaches at an inference cost comparable to zero-shot use.
That inference comparison excludes the conceptual distinction between fitting a reusable task and answering a one-off request. Stable tasks can amortise adaptation. Rapidly changing instructions may not. The reported efficiency should therefore be interpreted with the expected number of uses and the expense of maintaining the task state in view.
A hidden task vector can outlive its instructions
Because the vectors live outside the visible prompt text, an operator may not notice when they encode an outdated task definition. A revised label taxonomy or backbone update can change what the same vector does. Versioning needs to tie the vector to the task, training examples and model checkpoint.
The broader opportunity is a continuum between prompting and full fine-tuning: compile repeated demonstrations into a very small adaptation object. The associated risk is that compression makes the retained lesson harder to inspect. Useful deployment requires testing the vector’s current behaviour, not assuming its tiny size makes it semantically simple.
References
Questions, news tips or corrections? Contact [email protected].
