Why is it a closed loop?
When people look at a map, they don't take the next "screenshot" in their brains and then close their eyes to reason. The thought process keeps guiding our eyes: finding a clue, forming a hypothesis, and then going back to the image to verify it.
This closed-loop theory has a long-standing psychological basis. The classic book Eye Movement and Visual Concepts (1967) found that the same painting triggers very different eye trajectories under different problems: viewing is not decided unilaterally by the image, mission objectives and intermediate assumptions continue to be focused on focusing attention. The theory of sensory movement between vision and visual awareness (2001) further states that "seeing changes the understanding of how to master visual input with observation behavior; eye movements in natural behavior; eye movements (2005) indicate that people look back on the environment in real tasks".
This ability is closely related to the real application. Workshop work requires finding targets, checking differences in similar parts of a table; doctors searching and statistically counting suspect pathogens throughout the entire image; travelers crossing a route along a map and continually confirming that they are still on the right path. Each new discovery changes the area they see the next time, and the next observation in turn corrects judgments; understanding the big picture is far from enough.
Based on the above motivations, we have released a visual reasoning benchmark for ActiveVision that measures this "active observation" capability. ActiveVision has 17 tasks and 3 classes of capabilities:
• Distributed scanning: Find the whole number in the complete graph
• Sequential traversal tracking: Step by step along a structure to the bottom
• Visual attribute transfer: cross-regional memory and comparatively subtle visual attributes
The three classes of tasks in ActiveVision also correspond directly to the basic visual operations known in psychology. The four classes of tasks in ActiveVision are known as basic visual operations. The three classes of tasks in ActiveVision are known as basic visual operations. The three classes of tasks in ActiveVision are known as basic visual operations. The three classes of tasks in ActiveVision are known as basic visual operations.
Real applications of ActiveVision observation and the design of ActiveVision missions are shown in Figure 1.
This is close to 9 times the gap.
The results were clear: GPT-5.5 scored the highest score of 10.6% in all models without the use of external tools; 9/85); and the average accuracy of the three human participants was 96.1%; 94.1% were 97.6%.
Even the highest scoring model scored 0 points on 11 of the 17 missions.
Figure 2 Contrast the accuracy of pure CoT models with humans: the highest scoring model remains nearly nine times that of humans.
GPT-5.5 without thinking, answering directly, to use the highest reasoning intensity, the cost per question has increased by more than two quantitative levels, increasing its accuracy from only 2.4% to 10.6%; Fable 5 has an accuracy of only 3.5% at the highest reasoning intensity, while reaching 5.9% at the lowest reasoning intensity.
"Not long enough to think" cannot explain these results. A more direct indication is that the correct visual evidence has not been steadily reintroduced into the process of reasoning.
From the error model, the problem is consistent: the number of gaps when scanning globally, the sequential tracking when guessing directly from the start to the end, the language precursor when migrating attributes replaces true cross-regional comparisons. The model can generate an explanation that sounds reasonable, but it is unable to stabilize the closing loop of "obtaining evidence and preserving the middle state and returning to image verification".
The model saw the diagram, but could not continue to observe the objects in it.
Figure 3 shows the accuracy of the pure CoT model and the cost per question.
If you give it a code, can you look for it?
What if we gave the model code, drawings, measurements and other visual tools?
We gave the same set of questions to three coding agents, and the scores rose significantly: Fable 5 + Claude Code was 50.6%, GPT-5.5 + Codex was 37.6%, Opus 4.8 + Claude Code was 24.7%.
Agents are good at transcribing "seeing" into "counting": they try to solve them by splitting them by means of threshold splits, connectivity domains, path tracking, etc. However, once the actual texture causes the split result to break, or the tracker crosses a string, Agents are often unable to detect the output errors of these tools.
Furthermore, these improvements are not uniform. Three Agents reach 43% and 66% on attribute migration tasks that can easily be solved by tapping, template matching or geometry measurement; the Codex only works for 1/25, Opus 4.8 only for 3/25, and Fable 5 only for 10/25. The tool is only really effective when it can be reliably converted into a set of calculations.
The bottle did not disappear, but moved from "seeing the chart" to "checking to see if I saw something wrong".
Additionally, the cost of using Agent is not small: it takes an average of 12 tons 15 minutes per task, while the API equivalent costs $2.74 tons 7.63; humans take an average of about 34 seconds to complete without relying on any tool.
Tools will calculate, but they won't keep an eye on you.
Figure 4 Agent tooling performance and typical failures.
How do you create an exam that says "no if you can't see everything"?
In order to avoid drawing conclusions based on several hand-picked "drilling diagrams", each question in ActiveVision is generated by a determination process: the location, shape, curve, topology, and standard answer are all known; then GPT-image-2 redraws it into photo-level scenarios while maintaining the structure of determining the answer. Thus, each task can continue to be generated from new seeds, the answer is accurate, reliable, and does not allow the model to take familiar cartoon templates.
For example, a closed area can be transformed into fields and rivers from a bird's-eye view, arrow chains can be transformed into colored stones connected by footprints, and curves can be transformed into ropes on driftwood. Generators are responsible for "translating" known structures into materials, lighting, and textures; the solver must in turn restore the overall structure from the pixels.
At the same time, in order to prevent a model from looking at a single chart, it remembers all the contents, and no longer looks at the chart, all the elements in ActiveVision appear at random locations of continuous sampling, form-by-sample reproduction, and paths along random curves. The model cannot rely on grid coordinates, named shapes or fixed paths, and can only return to the image repeatedly in the process of reasoning, gradually searching for, tracking and verifying visual evidence, and cannot answer directly after a single glance in a compressed abstract.
ActiveVision focuses on whether sensory intelligence continues to participate in reasoning; describing a diagram is just the beginning.
The paper:
AlphaXiv：
The main page of the project:
The data collection:https://huggingface.co/datasets/activevisionai/ActiveVision
The rating code is:https://github.com/saccharomycetes/ActiveVision
Figure 5 shows ActiveVision's data generation process: definitive procedures, accurate answers and real-life photos.
Introduction by the author
Zhang is a Ph.D. student in Computer Science at the University of Southern California under the direction of Professor Willie Neiswanger. He received his undergraduate degree from Tsinghua University, where his research focused on multi-modal perception and reasoning and AI-for-Science.
Ceramics is a PhD student in Computer Science at the University of Southern California, and a Professor of Mathematics. He graduated from the ACM class of Shanghai Transportation University, where his research focused on multimodal learning, understanding and generation, and focused on the perception and generation of fine-grained visual perception of multimodal models.https://muzi-tao.github.io/
Wang is a Ph.D. student in Computer Science at the University of Southern California, taught by Professor Willie Neiswanger. He received his B.S. and M.S. degrees from Shanghai Technological University. His research interests include big language model reasoning, post-training, reinforcement learning, and AI-for-Science.https://shangshang-wang.github.io/