Core Conclusion: V-Rubrics divided 50,248 visual samples into 352,938 item-by-item checkable guidelines, allowing the facts, reasoning steps and task requirements in the image to be involved in the reward calculation, respectively. Based on the same SFT starting point and the same set of RL data, rubric GRPO reached 68.04 in Overall Avg. in general and knowledge assessments.
The reasoning process given by multimodal models can even be consistent, reading the wrong graph numbers, or writing objects that do not exist in the image. If the last answer is correct, the training method of evaluating the results may make these puzzles appear to be correct, but it is also rewarding if the actual reasoning is wrong.
In turn, the final answer is wrong, and does not mean that all previous observations are wrong. Those that are based on correct observations are still valuable training signals: retaining and strengthening these observations can help the model correct subsequent inferences on this basis. But this also raises a key question: how should the rewards be allocated, encouraged to do the right part, and directed to the model to correct the wrong linkage when correct observations and wrong inferences are intertwined in the same paragraph of the answer?
A team of researchers from the S-Lab, A*STAR and UIUC have proposed V-Rubrics to address the question of credit assignment using rubrics on a case-by-case basis. Visual facts, reasoning steps and task requirements are listed as criteria for evaluation, and the scores of these individual scores will be used in subsequent GRPO reinforcement studies. The paper has been received by EMNLP 2026 and the code and data are now open source.
V-Rubrics: Visual Faithfulness through Rubric-Based Reinforcement Learning
It was written by Shulin Tian, Minglun Li, Yuhao Dong, Hao Ding, Jiarui Yao, Haiwen Diao, Jingkang Yang, Hongyuan Zhu, Ziwei Liu.
Address of the paper:https://arxiv.org/abs/2608.25580v1
The main page of the project:https://shulin16.github.io/v-rubrics/
The code repository:https://github.com/shulin16/v-rubrics
The data collection:https://huggingface.co/datasets/v-rubrics/v-rubrics-50k
Right observation and wrong inference
Why do we evaluate them separately?
The paper gives an intuitive example. When asked how many people were born after the end of World War II, the answer-level GRPO model accurately identified four people, but then estimated the age of one of the men to be over 70 years old based on the appearance of one of the men, and claimed that he was born before 1945 and ultimately answered three people.
If the final answer is only calculated, the training can only know that the whole question is not correct, but it is not possible to make more specific judgments: identifying whether the four-person puzzle is based on an image, and subsequent inferences need to be corrected. For graphs, geometric questions or document questions, such mixtures are equally common, and a misreading or misconstrued inference may change the final result.
Writing visual evidence in rewards
V-Rubrics divides the guidelines into three dimensions around the formation of a visual response:
Visual Faithfulness (VF) Fact-based checks. Objects, attributes, relationships, numbers, visible text and chart values mentioned in responses should be supported in images.
Reasoning Consistency (RC) checks whether the inference is valid. Even if the image information is read correctly, subsequent conclusions need to be able to be drawn from these facts.
Instructions follow the instruction following, IF, to check that the task has been completed, including answering form, subject requirements and other explicit constraints.
The same visual problem is broken down into three types of VF, RC, IF atomic norms, each of which is independently judged and weighted.
The design of the rubric is also important: each rubric examines only one atomic element and requires that it be expressed as briefly as possible, independently derived and understandable, and with important labels and corresponding weight. Essential, Important, Optional, respectively, must be answered, important but not necessary, and can be supplemented with content; Pitfall specifically identifies such common illusions and reasoning errors.
How to make automatic scores verifiable?
In geometrically titled examples, the answer should be correct and the answer should not provide the rater with how much additional information; the object is checked if it specifies the length of the margin to be read, the formulae to be applied and the target area respectively. The paper requires the rubric to point to specific observable facts, and to list the object and the set conditions in the single rule of thumb, while covering key observations and intermediate reasoning.
These checks are also important. The visual questionnaire uses the priority VF ≫ RC > IF to prioritize visual fact, then consider reasoning and instructional constraints; the vertical weight of Essential、Important、Optional is 1 to 5, and Pitfall is marked as −1 or −2.
When generating guidelines, the model reads the prototypes, instructions and reference responses, and writes the facts that need to be verified into a rubric; when training, the verifier only reads the generated responses with these self-contained guidelines, rather than directly reading the prototypes.
Fifty thousand questions
How to provide 350,000 specific feedback?
V-Rubrics 50K samples were taken from 17 public visual data sources, including charts and document questionnaires, diagrams, visual mathematics, statistics, educational questionnaires, and general visual reasoning, and were initially screened for effectiveness, content quality, and repeatability.
To determine whether the subjects still had a training value, the team had the model monitored to respond eight times to each candidate sample: all of the subjects were eliminated because the final answer rewards made it difficult to distinguish between these responses; the remaining samples were divided by the difficulty in proportion to the responses.
Of the 50,248 samples that were ultimately retained, the hard-level problems accounted for 36.1%, the medium for 50.4%, and the simple for 13.6%.
The Gemini-3-Pro subsequently generated the VF, RC, IF guidelines in accordance with the Unified Image Conditioning Protocol, for a total of 352,938 rubrics. Of these, VF is 209,436 articles (59.3%), RC is 101,369 articles (28.7%), IF is 42,133 articles (11.9%). Nearly six guidelines are used to check visual fact, allowing the camera to accurately understand whether image frames are the main monitoring content.
Each sample contains images, subjects, reference answers and corresponding guidelines, which can be checked separately for compliance with the requirements during training.
V-Rubrics 50K data sources are distributed according to VF, RC, IF guidelines
After the demolition of the rating
How do you get into reinforcement learning?
Each score is only the first step. If all the guidelines are combined into a total score and the entire response is updated with this score, the difference between the judgments will still be compressed when credit is allocated. To solve the shrinking problem of this reward, V-Rubrics therefore adjusted the reward content and credit allocation at the same time.
In the main experiment, the correctness of the answer and the satisfaction of the rubric were weighted with a semantic reward of 0.5 while retaining the standard format reward.
For the same question, the system produces more than one final answer and scores for each rubric. For example, a set of answers may not have been correctly concluded, but only a portion of the answers correctly read the numerical values of the graph; each comparison can retain this difference. On the contrary, if all scoring answers score the same on a given criterion, this item does not produce a group relative gradient.
Let the training signal match the response forecast with the evidence
When constructing the training advantage, the answer score is standardized with the points scored in the same group of rollouts respectively. The final answer's signal covers the entire response, while each rubric retains an independent advantage signal and participates in updates according to the corresponding authority.
Next, the verifier returns the corresponding original statement for each judgment, and the system then places the sentences in the response by blurring matches. The advantage of each rubric only serves as a foreword from the start of the response to the end of the corresponding statement. Subsequent content no longer uses this advantage.
For hallucinations or reasoning traps that have been confirmed, Pitfall initiates a denial mechanism for semantic rewards, eliminating the corresponding response credits and positive rubric credits.
V-Rubrics data builds and reinforces learning processes
Individual feedback
What skills have been improved?
In the two main groups of assessments, rubric GRPO scored higher than the average score of SFT and response grade GRPO. Of these, the Overall Avg. for general and knowledge assessments reached 68.04, and visual mathematics, graphs and logic assessments reached 62.45.
To make comparisons focused on the reward mechanism, the two RL schemes use the same training starting point: we use Qwen3-VL-8B-Instruct as the base model, use the OpenMMReasoner-SFT-874K to monitor the subtlety, and then start from the same SFT checkpoint, respectively training response level GRPO and rubric GRPO. Both use the same set of V-Rubrics 50K assumptions, and maintain the same numerical budget and optimizer settings.
Compared to SFT, the overall performance of the two groups of ratings improved by 3.11 percentage points and 4.00 percentage points, respectively; and further by 1.79 percentage points and 0.51 percentage points compared to the response grade GRPO. Two Overall Avg. benchmarks for all ratings in their respective tables were calculated.
First, change the reviews, and then change the distribution of the rewards.
In general and knowledge assessments, only rubric scores were added, while scores were still synthesized by scalar sequence-level aggregation, the Overall Avg. rose from 66.25 to 67.74, an increase of 1.49 percentage points.
On this basis, the scores were individually standardized and the signals were assigned to the corresponding response lead, averaging 68.04 points further. The new 0.30 percentage points reflected the common role of component-wise standardization and lead positioning.
MMMU series: Knowledge reasoning is the main source of growth
Improvements in general and knowledge assessments are mainly due to the MMMU series. This type of subject requires models to connect the local information in images with academic knowledge, a correct answer often contains multiple interdependent judgments, and provides clear examination objects for fine-grained feedback.
Compared to the response grades GRPO, MMMU Val, MMMU-Pro and MMMU-Pro Vision, which improved by 2.56%, 2.43 and 2.60 percentage points respectively, Knowledge Avg. rose from 59.35 to 61.88 points.
Table 1: Complete results of the benchmark for general visual abilities and knowledge and academic reasoning
Visual mathematics, graphs and logic: further up to 62.45
In visual math and logic tasks, MathVision improved from 56.71 to 58.88, LogicVista from 60.63 to 62.42; DynaMath Worst and WeMath Loose also improved by 1.20 and 1.43 percentage points respectively.
The overall subtasks, Math Avg. increased from 63.19 to 63.63, Chart Avg. increased from 58.81 to 59.51, and Overall Avg. increased from 61.94 to 62.45.
Table 2: Complete results of visual mathematics, charts and benchmarks for logical reasoning
So, back to the specific answer, where is the difference?
In the previous human-like problem, the model after rubric training supplemented the estimate between age and year of birth; in another functional image question, the answer-level model misread the function value at 2 for the first time as x = 4, while the model after rubric training re-read the image in stages, obtaining x = 2.
Two types of examples: from unsupported age inferences and corrective intermediate reasoning from incorrect image readings
Code and training data are open source, and you are welcome to join us for discussion.
Author's note
She is currently a Ph.D. student in Computer Science at the South Pacific University of Technology, where she is mentored by Prof. Zhongqing and Dr. Zhongqing. Her research focuses on multi-modal intelligence and visual reasoning, with a particular focus on long-term multi-modal understanding and tool enhancement reasoning. She has received A*STAR computing and information science fellowships and has been nominated as Outstanding Reviewer for CVPR 2026 and ICLR 2025 SCOPE Workshop.
