When an object leaves a camera’s view, a robot must carry forward a belief about its location. That requires both storage and a model of motion. A representation can retain a great deal of information yet still predict the wrong location because its allowed dynamics exclude what the object actually does.
The comparison between FloWM’s grid memory and a small object tracker isolates this distinction in a controlled block world. The tracker receives privileged observations, so the experiment is not a fair contest between complete perception systems. It is a way to ask which assumptions about state and motion explain the larger model’s apparent memory performance.
The readout changes the score
An October 13 comparison probes FloWM's spatial grid against an oracle-fed Kalman tracker in the model's block world. Results depend on how the grid is read. The tracker also handles diagonal motion that lies outside the grid's built-in velocity choices much better. Read the research paper.
FloWM represents objects in a feature grid moved through five built-in velocities. A node tracker represents position and velocity numerically. The paper reports that a single-cell probe locates 66% of out-of-view blocks, while FloWM’s own probe locates 89%. How information is read therefore changes what the memory appears to know.
The authors also observe higher performance for in-view objects even when no new frame is supplied, attributing this to write-back of the model’s predictions. That makes the evaluation pipeline part of the explanation. A score can reflect repeated self-updating and the chosen decoder, not merely the fidelity of a stored observation.
The memory budget grows with the chosen dynamics vocabulary
The paper’s faster-motion example highlights another consequence of the representation. Adding an allowed speed lets the tracker recover strong performance, while representing the same expanded movement vocabulary through shifted grid copies would multiply storage. The cost is linked to which transitions the representation can express, not just to how many objects exist.
That is why the compact tracker is an informative architectural baseline despite its privileged inputs. It demonstrates one way to decouple numeric state from a small enumerated set of motions. The unresolved perception and noise problems remain substantial. A learned tracker still has to identify objects, estimate velocities through gaps and handle collisions or bounces. The comparison should therefore guide diagnosis rather than serve as a blanket victory for hand-built systems. It shows that a memory design can be large partly because of its transition machinery, while the underlying object state being remembered is much smaller.
Diagonal motion exposes the transition assumption
In the reported diagonal-motion test, outside the grid’s five built-in velocities, FloWM finds 11% of blocks even after fine-tuning, while the unchanged oracle-fed tracker retains 93%. The tracker reaches FloWM’s original range with roughly 14KB of state against 5.6MB for the grid.
These numbers isolate a useful representational advantage under privileged sensing. They do not price the missing learned detector, nor establish robustness to real occlusion, noisy velocity estimates or complex contact. The paper explicitly leaves a learned node memory to replace the oracles and handle those harder conditions.
A privileged baseline can be an explanatory tool
Giving a baseline perfect inputs is inappropriate if the headline question is which complete deployed system wins. It can be appropriate when the question is why a particular representation fails. Removing perception error lets the experiment reveal a transition-model limitation that more storage would not cure.
The wider lesson is to decompose memory claims. Observation, retained state, prediction and readout each deserve their own intervention. An expressive neural memory may still inherit a narrow hand-designed motion vocabulary. Finding that boundary is more useful than treating every out-of-view error as evidence that the memory bank needs to become larger.
References
Questions, news tips or corrections? Contact [email protected].
