EvaluationsThe robot encoded the right instruction. Its hand still went elsewhere.
A new study separates information inside a policy from information that actually governs its actions.
Evaluations
The tests and evidence that separate a stronger model from a better-looking score.
Benchmarks · Reliability · Cost & performance
9 articles
Evaluations
EvaluationsA robot benchmark begins before the robot moves
Inspect Robots makes setup and termination programmable. Those unglamorous steps help determine what a performance score means.
EvaluationsThe right tool depends on the body holding it
HumanoidToolBench separates recognising a useful tool from carrying, positioning and successfully using it.
EvaluationsA robot with touch sensors may still be solving the task by sight
OpenViTac’s paired simulation and physical tasks make it possible to ask what tactile input contributes, rather than just whether it is present.
EvaluationsOpenAI and Ironclad: One missed business rule can erase an AI agent’s progress
Their contracting-workflow test shows why a better average score is not the same as a workflow a company can switch on. Mandatory rules need their own acceptance test.
EvaluationsSurge AI: A model can get the spreadsheet math right and still give the wrong answer
GDP.xlsx tests whether models find the evidence across a workbook before calculating. A missing sheet can matter more than another improvement in arithmetic.
EvaluationsMercor: Beating accountants on a test is not yet proof of cheaper accounting
Its bounded comparison favors AI on structured tasks. The next question is whether that advantage survives ordinary workplace context, review and exceptions.
EvaluationsHarvey’s AI test: One invented fact can cancel an otherwise complete answer
Artificial Analysis and Harvey now gate headline credit on grounded answers. That separates two problems model teams must solve: missing requirements and unsupported claims.
EvaluationsFactory: Your old test cases may be the key to trusting AI-written repairs
Legacy-Bench exposes a gap between an agent saying it is done and a repair passing independent checks. Preserved outputs and edge cases can matter as much as a stronger coding model.
No articles in this topic yet
Explore the other topics in this desk.