It's the first time in the history of the world.
Imagine a task: handing the camera over to AI and getting it into a virtual room you've never seen before. It doesn't have a pre-determined object coordinate, nor a well-arranged route to visit. It needs to decide where to turn, where to move, when to take another picture, and finally tell where each target object is in 3D space.
Link to the paper:https://arxiv.org/abs/2610.26659
The code link:https://github.com/JesseZZZZZ/SpatialScout
The data collection links to:https://huggingface.co/datasets/jessezhaoxizhang/SpatialScout
SpatialScout turns this exploration process into a research task that can be recorded, rated and compared. Each observation will leave a mark on the final object-level map as to whether it is useful, whether different points of view can be mutually validated, and whether the coordinates submitted are reliable.
Figure 1 A space exploration consists of a combination of camera action, multiple viewpoint observation and object position estimation.
Let the model personally select the next observation: starting from the RGB image, using camera controls and geometric tools to estimate the object center. Exploration strategies and spatial inferences are examined in the same closed loop of interaction.
Moving towards physical AIWe need to build a sense of space first.
People walk into a room, look around, circle around, and usually have a coherent understanding of the overall layout: which side of the table is next to the wall, which side of the chair is next to the table, and which direction the object is heading after turning around.
Physical AI also needs this foundation. Smart entities must organize discrete visual cues into stable spatial relationships if they are to operate continuously in real environments. Even basic locations and layouts are difficult to determine through observation, and subsequent action planning, object operation and environmental interaction also lack reliable references.
To build a common ground for this basic capability
The official evaluation selected 80 indoor scenarios and 3,140 target examples, from iTHOR, RoboTHOR, ArchitectTHOR, ReplicaCAD and ProcTHOR. The room types include kitchens, living rooms, bedrooms, bathrooms, apartments and multi-room dwellings. Fixed scenarios are set up with the initial camera, allowing different models to face consistent mission conditions.
Figure 2 Data set objects category word cloud. This figure is based on the statistics of 3,440 instances from the full 90 scenarios directory, which are summarized by the original classification rules to 143 categories; the official results use 80 scenarios evaluation protocols.
Chairs, tables, cabinets, books, lamps and everyday objects all make up these environments. The size, positioning and visibility of objects varies, and models need to constantly adjust their perspective to make connections between local details and the overall layout of the room.
Take a picture of it.Becoming a Deliverable Coordinate
At the start of each task, the model receives a list of target objects with the initial image. The object library tells it which objects to look for, but does not reveal the coordinates. The model can actively shoot, move the camera, read the camera's participatory positions, and hand over the multi-angle correspondence observations to the triangular measuring tool; after the location is confirmed, it can be submitted in batches at any time and continued to be modified. Each task has a maximum of 100 model interactions.
At the same time, the score focuses on "finds incomplete" and "puts out incomplete": the average IoU of all targets and the four metrics for each of the submitted targets, as well as the ratio of IoU to more than 0%, 50% and 90%. The non-submitted targets are marked as zero in all object metrics.
Present model performanceA map of space that is still intact
Table 1 averages the macro by task based on the trajectory at the end of the normal execution of the finish_mapping.
Compared to the current seven models and eight configurations, Opus 5.5's mission average IoU is 41.71%, Astra xhigh is 33.16%, and Sonnet 5.5 is 32.28%.
Figure 3: From accurate layout to object absence, location deviation, and image reconstruction intuitively present different forms of success and failure.
The problem may arise after "observation".
Seeing an object does not mean that its spatial position has been determined. If there is no consistent correspondence between observations, an increase in the number of images can only lead to more local impressions. The model also needs to identify the same object at the same location from different angles and select a shot point that provides effective visual differences in order to establish reliable geometric constraints.
Figure 4 shows the call distribution of different types of tools during the exploration process, showing the scheduling differences between observation, camera movement and mapping operations.
Observing inputs and charting calculationsWe need to really connect.
The tool called the distribution demonstrates a phenomenon of interest: some models observe repeatedly in exploration, frequently move cameras, and subsequent mapping operations are not fully followed. They may have collected a number of images, but fail to effectively organize information across perspectives, geometrically implementing "seen" as "located".
In particular, triangular measurements test precise spatial perception. The input needs corresponding image points from different shooting locations; object centers mark deviations, obscurations, and visual impairments, all of which may affect the estimate. In this regard, observing accuracy, timely calculation, and verification of results are more important than simply increasing the number of shots.
Geometric measurements support final submission
Of the 80 joint missions, 99.33% of the objects eventually submitted by Astra had previously received successful triangular measurement support; DeepSeek was 63.48%. This difference provides an analytical cut-in to "observing how to convert to positioning evidence".
These trajectory clues support an improved direction: enhancing the accuracy of cross-sectional correspondence and enhancing the synergy between observation, computation and submission. The number of tools itself cannot directly measure the quality of observations, and the above analysis is descriptive evidence.
We have to break down failure to find the next step.
The role of SpatialScout is also to make the process traceable: missing objects may point to insufficient coverage, distorted layouts may point to unstable coordinate inference, and low scores after large numbers of observations suggest the need to examine visual correspondence with geometric tools. Researchers can use these clues to design more targeted exploration strategies.
The quality of a mapIt's also necessary to look at the cost of exploring.
Continuous shooting, long-running context reasoning, and multi-cycle tool interaction all consume resources. SpatialScout will reconstruct IoU with the average cost per mission in the same chart, using the Pareto edge to help observe quality choices under different budgets.
In Figure 5, the distribution of the cost of mass pumps is shown. The average cost is in dollars, the horizontal axis uses a pair of scales, and the parallel line connects the front of the Pareto configuration.
According to the current stored price and token record estimates, the average for Astra is about $2.47/task, the average for Astra is about $3.06/task, and the average for Opus 5.5 is about $27.90/task. DeepSeek V4.1 Flash averages about $0.027/task, and its corresponding positioning scores are low.
Fees are calculated using OpenRouter fixed-price snapshots, and only for the task averages for tokens that record complete tasks. The IoU in the cost graph is a micro-indicator aggregated by object, while the prefix table is a macro-indicator for the task averages. Fees are estimates under the price snapshots, and do not represent actual bills.
Make the next observation more valuable
From selecting a new perspective, to identifying a set of spatial coordinates, to creating an object-level map that is available, active spatial understanding requires a whole chain of capabilities. SpatialScout provides fixed tasks, interactive tools and scoring processes that allow researchers to access their own models, and compare new perspective planning, multicolored reasoning, and geometric tool strategies in the same environment.
This benchmark will help more research to focus on the space needed for real action: letting the intelligent body know where it is, where the object is, and then deciding how to observe and act next.