A response can be accurate but badly justified, or complete but poorly grounded. Collapsing a rubric into one reward number hides which property the learner should change.
CoCA's October 13 paper derives credit from criterion-level outcomes and distributes it using criterion-related changes in token probabilities. Its evaluated medicine and science benchmarks improve over the comparison rubric-RL approach. Read the research paper.
A careful rubric can lose its detail in training
Open-ended answers rarely have a single mechanically verifiable definition of correctness. Evaluators therefore use rubrics covering properties such as completeness, evidence and explanation. That creates a richer description of quality than a single score. Yet many training pipelines immediately compress the description back into one reward and apply a uniform advantage across the response. The evaluator has identified different kinds of success and failure; the optimiser receives a much coarser message.
CoCA asks whether the learning process can preserve more of the rubric's structure. Its focus is credit assignment: deciding which parts of a response should receive which kind of reinforcement.
Outcomes decide the credit; criteria guide its location
The method derives criterion-level credits from group-relative outcomes. It then distributes those credits across tokens using changes in the model's distribution induced by the criteria. This separates two jobs. Observed criterion outcomes ground the amount or direction of credit, while rubric-related information helps identify where in the response the relevant behaviour appears.
That is different from treating a rubric-conditioned answer as an imitation target. A teacher can produce a response that sounds more compliant with instructions without demonstrating that it improved the criterion being rewarded. CoCA is designed to keep the signal connected to assessed outcomes while still providing more local guidance than one response-wide number.
What survives the collapse into a scalar
Consider an answer with a correct conclusion and an unsupported explanation. A uniform update risks reinforcing both together if the total reward is positive. The opposite problem occurs when one failure causes an otherwise useful passage to receive negative pressure. The proposed decomposition seeks to distinguish those cases rather than requiring the model to infer the entire correction from a single aggregate judgement.
The authors report an average improvement of 4.2 percentage points over standard rubric-based RL across their medicine and science benchmarks. This is an evaluation of a training method under the paper's tasks and graders. It is not evidence that the resulting model is qualified to deliver clinical advice or that the grader's criteria capture every aspect of scientific reliability.
Conflicting signals are visible before the final score
The authors measure cases where the response-level advantage and at least one criterion’s credit have opposite signs. Such conflicts appear in roughly half of the analysed medicine and science rollouts. They also find frequent misalignment between distillation and outcome-grounded logit gradients for criterion-token pairs. These diagnostics support the claim that flattening feedback can send the wrong local learning signal.
The default experiments use Qwen3-8B with a gpt-oss-120b rubric judge and train separate medicine and science policies. Evaluation includes both corresponding task splits and additional domain benchmarks. This provides more context for the reported average improvement: the method is tested across several assessments, but still depends on model-based criteria whose reliability must be examined independently of the optimiser.
Local feedback still needs validation
Associating a token with a criterion is not the same as proving that the token caused a result. A style phrase may correlate with a high score while contributing little factual value. Evaluation should therefore inspect whether the intended property actually changes: does grounding improve, for example, or does the answer merely adopt wording associated with grounded responses? Independent outcome checks are valuable when the training grader can be imitated.
Better judgement at the evaluation stage is partly wasted if the update stage erases its distinctions. A useful rubric needs a route into optimisation that retains the information responsible for its usefulness, while keeping the connection to actual task outcomes visible.
References
Questions, news tips or corrections? Contact [email protected].
