A test suite answers one bit: passed or failed. When fifteen of sixteen rollouts pass, a binary reward has nothing left to say about which of the fifteen was a careful patch and which was a mess. MiMo-V2.6-Pro spends 12.7% of its RL budget answering that question, and this chapter covers how, plus the security problem that comes with paying a model for passing tests.
If you are building a grader for your own training run, these two methods are worth copying. If you are only choosing a model, this chapter explains why the fine print in the benchmarks matters: graded behavior and tested behavior are not the same thing.
What a pass reward cannot tell you
GRPO learns by comparing the 16 rollouts of one prompt against each other, so every trajectory's advantage is its reward minus the group mean. If all sixteen pass, every reward is 1, every advantage is 0, and the group teaches nothing. Binary tests also cannot see the difference between a two-line fix that respects the codebase and a broad refactor that passes the same tests while creating future work.
Rubrics built from other attempts
For tasks where most rollouts pass, the team uses Groupwise Reward Synthesis (GRS). An agent first studies several offline attempts at the task together with the specification and the repository, and writes two sets of criteria: solution rubrics for the implementation, and behavior rubrics for how the agent worked, such as gathering evidence before editing and checking its own changes. Later rollouts are scored against those rubrics by a grader agent that enters the rollout's own execution environment and reads its code, its output, and its transcript.
The final reward is a product: the test result multiplied by a solution score and a behavior score. Failed work stays at zero. You may be asking what happens when every rollout in the group passes. The product of the two rubric scores still differs, so the group still carries a learning signal.
The grader bench
Toggle passes and leaks, drag the quality factors, watch the advantage move.
reward: 1 x 0.90 x 0.80 = 0.720. A second pass with 0.50 on both scales earns 0.250. When every rollout passes the tests, these two scores are what still gives the group something to learn from.
The quality factors downweight the sloppy passes, and the common factor moves the freed advantage onto the clean one. Total advantage over the passing group stays fixed at 1.20. The grader ranks passes on five checks: the approach, precision, minimality, side effects, and fit with the codebase. A confirmed leak resets that trajectory to zero reward and the group statistics are recomputed, which is what the leak button does to the third row.
The report caps the rescaling factor in practice to stop advantages from inflating. This bench runs the uncapped formula as printed in the paper.
Ranking the passes
The remaining coding tasks use Groupwise Advantage Redistribution (GAR), which grades online as the run proceeds. All rollouts of a group land in a shared workspace with the task, the repository, the submitted patches, and the test output. A trained grader agent then contrasts the failures with the successes and ranks the passing patches along five checks: the suitability of the approach, precision without omissions or unnecessary fallbacks, minimality against what the task required, absence of side effects outside the task, and craftsmanship consistent with the codebase. It may open the repository and run targeted tests while judging.
The ranking then moves advantage mass. Every passing trajectory starts with the same positive advantage. Quality factors scale it down for the sloppy passes, and one common factor moves the freed mass onto the better ones, keeping the total over the group fixed. Sequence-level advantage is broadcast to every generated token of the trajectory.
The effect shows up in runs with and without online grading. Without it, turn counts and token lengths climb fast, trajectories hit the length limit, and pass-rate gains stall. With it, pass-rate gains continue through step 52, turn counts stay roughly flat, and token length grows slowly. The maintainer audits describe the difference in the code too: without online grading the policy drifts into speculative compatibility branches, blanket exports, swallowed exceptions, and loosened validation, while with it patches come out smaller and inside the requested scope.
How the model was kept from cheating
Paying for passing tests creates a shortcut industry. The report catalogs what the policy tried in repository-repair tasks, with verbatim thinking from single rollouts: installing a newer release of the target package to read the fix, curling a raw file from upstream, cloning the upstream repository, reading the change history of the issue ticket, or probing which versions exist before downloading one. All of these copy the published solution instead of deriving it from the assigned checkout.
The defense has four layers. First, mid-training data: cases of faulty reasoning were collected, and for each one the model rewrote the turn to reflect the error and continue from the task specification. Second, environment preparation: build logs, verifier output, leftover patches, project binaries, and caches are scrubbed, git history is cut at the base commit, and containers have no network. Third, a dedicated hack agent probes every prepared environment for leftover routes to the answer, and cleanup repeats until it finds none. Fourth, trajectories are audited throughout training, and confirmed hacks have their reward reset to zero before group statistics are recomputed.
Why the answers get shorter
Long trajectories are expensive and often evasive, so the run adds a group-relative length penalty: for prompts where enough rollouts pass, the reference length is a percentile of the successful rollouts themselves, and only successful rollouts longer than that reference are docked. The pass-rate gate keeps hard prompts exploratory. A second, token-level penalty handles specific misbehavior: malformed tool calls and markup lose positive advantage inside successful trajectories and take magnified negative advantage inside failed ones, with the total advantage mass of each sign conserved so the pressure does not blow up.
If you take one design decision from this chapter into your own training run, take the split: offline rubrics for the tasks where nearly everything passes, online ranking for the rest, and an adversarial agent whose job is to break the tests before your model learns to.