Verifiable Rewards

Verifiable rewards are a class of reward signals used in reinforcement learning for LLMs where the correctness of a model’s output can be confirmed objectively against a known ground truth. Unlike preference-based rewards – which rely on human raters or a trained reward model to judge which output is better – verifiable rewards can be checked automatically and unambiguously. This makes them a highly reliable and scalable source of training signal for tasks that have definite right answers.

How it works

During online RL training, the model generates one or more completions for each input prompt. In a verifiable reward setting, each completion is evaluated by a deterministic checker rather than a human or a learned model. For a math problem, the checker confirms whether the final numerical answer matches the reference solution. For a coding task, the checker runs the generated code against a test suite and reports whether it passes. For a structured task such as grounded question answering, it checks whether the model’s response is consistent with the provided source. The binary or graded score returned by the checker is used directly as the reward signal to update the model. Because the scoring function is objective and automated, verifiable rewards can be applied at large scale without the cost or latency of human annotation.

What it is used for

Verifiable rewards are used to train LLMs on tasks requiring precise, checkable outputs – most prominently mathematics, programming, and formal reasoning. They are a central component of reinforcement learning from verifiable rewards (RLVR), the training approach behind many state-of-the-art reasoning models. Because verifiable reward signals are noise-free and consistent, they tend to produce more stable training dynamics than preference-based signals. However, they are inherently limited to domains where ground-truth answers exist; for open-ended or subjective tasks, preference-based or model-based reward methods remain necessary.