“ Navigation and perception break down at the same time.
There is a classic dilemma in the field of robotics.
It makes it relatively easy for robots at the abstract level to understand string instructions, making it increasingly mature in the physical world to execute string operations, but connecting the two seamlessly in an open, dynamic, unstructured real environment always creates a huge gap between laboratory and industrialization.
Recently, two studies conducted jointly by the StarSource Intelligence team and relevant universities and research institutions, called DA-Nav (Direction-Aware Navigation) and AdaOcc (Adaptive 3D Occupancy Prediction), have uncovered key technological pathways to practical use of AI.
A common keyword for the two works is the Ada Adapter. DA-Nav addresses how to convert rough human navigation instructions into robotic spatial decision-making, while AdaOcc focuses on how to make 3D perception models adapt to different computing abilities and real robot platforms with different sensor configurations.
They originate from different technologies, but they all point to the same premise: real-world physical intelligence must learn to adapt dynamically in uncertain environments.
In other words, what robots really need is spatial intelligence.
01.
DA-Nav: The problem of turning navigation into a mirror image option
When it comes to robotic navigation, the first reaction many people have in mind is that SLAM allows robots to map, reposition, and ultimately plan routes. This set of technical routes is quite mature in indoor structured environments, and once expanded to urban-level outdoor scenarios, problems arise. In open environments such as cities, parks, parklands, and other dynamic factors such as road construction, vehicle parking, pedestrian traffic, and other dynamic factors can cause rapid map failure. In addition, the cost of collecting and maintaining high-precision maps is expensive and difficult to cover wider open scenarios.
In recent years, researchers have begun attempts to get robots to use the navigation information provided by commercial navigation software such as Google Maps and Goode Maps. This navigation information is clear enough for humans, but far from enough for robots, because robots need to know exactly where to turn left and right in the image and what trajectory to follow.
They didn't try to make the model directly output continuous three-dimensional coordinates or underlying control instructions, as existing visual language models were prone to spatial hallucinations that misplaced the object's position and distances.
The solution to DA-Nav is to redefine continuous navigation planning as discrete image option tasks.
The specific approach is to divide a virtual grid of 29 rows of 24 in the front image taken by the robot, and the model is to select the best target point sequence from the grid. The frame in the center of the image to the right may mean a straight line curve, and the grid on the left edge may represent a curve that turns to the left. This decentralized processing, equivalent to adding structured geometric constraints to the model's decision space, makes it less difficult to predict chaos.
The method has both the advantage of VLM in image-spatial reasoning and the avoidance of spatial hallucinations and unstable closed-loop behavior by structured constraints.
More importantly, the DA-Nav introduced a mind-chain reasoning mechanism. The model is not a step-by-step approach to selecting points, but first determines whether it deviates from the intended path, then decides what action should be performed, and finally produces a specific trajectory point. This kind of pre-evaluation, decision-making, and post-executing structure allows the model to have the potential to self-correct when deviations occur.
Failure recovery is the easiest part of traditional simulation learning. Most navigation models are trained with perfect human instructional data, and they only see the right path, without seeing how to correct the error. In a real environment, once a robot deviates from its intended trajectory due to bumps, obstructions, and dynamic obstacles, it will quickly slip into a state that has never been seen before in training, and errors will accumulate to a certain extent out of control.
To solve this problem, the research team specifically constructed a ReDA dataset. They intentionally injected interference into the simulation environment, allowing robots to deliberately deviate from the path and then record how the experts corrected it. The entire dataset contained about 286,000 sequence samples, of which about 128,000 tonnes were recovered behavioural data, and the model learned how to pull back in the face of deviation after training with a mix of 158,000 tonnes of normal navigation data.
In 239 long-distance closed-loop orbital tests, DA-Nav's navigation success rate was 59.00%, and the correction success rate after deviation was up to 98.15%, compared to the one-digit success rate of other contrasting schemes.
Even more convincing was the zero-sample migration of real-life scenarios. The team deployed trained models directly on manufacturers' quadruped and humanoid robots, without making any tweaks to real data, and completed more than 1,200 meters of stable navigation in different urban environments.
This means that the model learns not only the surface characteristics of the simulated environment, but also the ability to migrate spatial reasoning and bias correction.
02.
AdaOcc: Transforming 3D perception into a soft-core capability
In order for a robot to complete a task like taking a remote control from the living room and putting it on the coffee table, it needs not two-dimensional images, but a three-dimensional spatial model with rich geometric and semantic information.
3D Occupancy, which has received increasing attention in recent years, is an important way to solve this problem. Compared to traditional point clouds or depth charts, it not only describes the locations occupied by objects, but also further assigns a semantic category to each spatial location, forming a complete 3D digital world.
However, the existing Occupancy method still maintains a significant gap between the actual deployment and the robot system. Different robots use different sensors, quadrupeds may have only one front-facing camera, humanoids may be equipped with multiple RGB-D sensors plus laser radars, and rotary robots may have a somewhat weaker load capacity than humanoids. In addition, different tasks require completely different spatial understanding precision requirements, navigation emphasizes speed, and operations require more precise environmental modeling.
A perceptual module that can only work under a specific configuration is far from the actual deployment.
AdaOcc's design motivation is to solve this problem of adaptability.
It first constructed an adaptive geometrically guided binary branch encoder. In which the visual branch processes RGB images, the geometry branch processes point cloud data collected from depth estimates, depth cameras or laser radars. Regardless of the geographic source, it is uniquely expressed as a point cluster under the camera coordinate system, sharing the same point cluster interface after integration processing, thus dissolving with the specific sensor.
In the prediction phase, AdaOcc does not predict dense object grids, but instead produces sparse semantic points. These points carry location information and semantic tags, and are then quantified as standard assessments. The use of sparse point expression has two advantages: high computational efficiency and the ability to dynamically adjust accuracy and latency by controlling the number of query points.
In the course of training, AdaOcc adopted a progressive query learning strategy. In the early stages of training, fewer query points were used to gradually increase the number of upper limits as the training progressed. With this course training model, it was possible both to pursue optimum accuracy on a budget when reasoning, and to meet low delay scenarios with fewer query points or shallower levels of decoders.
For example, with a 3-layer decoder alone, a delay of 58.2 milliseconds is considered, while with a full 6-layer decoder, a delay of 64.4 milliseconds is used, while with a full 6-layer decoder, a delay of 1710.6 milliseconds is used, and the mIoU is raised from 49.46 to 57.97, so that the same model can be flexibly weighed without re-training.
Another innovation in AdaOcc is Containment Loss. A common problem in point cloud reconstruction is that predictive points will float near the surface of an object, causing the three-dimensional boundary to be inaccurate enough. AdaOcc introduces a clever constraint that requires predictive points to fall as far as possible inside the real-world area of occupation, rather than hanging in blank space, effectively reducing boundary errors and improving overall geometric consistency. For robots, this means that built-in space maps are more stable and more suited to cooperate with basic environmental models for navigation and operation tasks.
The results of experiments also prove this. In the complete Occ-ScanNet dataset, the mIoU of AdaOcc is 59.67, and the IoU is 65.29, 7.84 and 2.46, respectively, points higher than the previous best method SplatSSC. This improvement is a significant leap in the field of 3D perception.
The team also verified the multi-tasking applicability of real-world scenarios. The same pre-trained model supports building obstacle maps in navigation, positioning target objects in operational tasks, semantic target navigation in mobile operations plus crawling, involving multiple forms such as quadruped machine dogs, humanoid robots and robotic arms.
03.
The intersection of perception and action
If you look at the DA-Nav and AdaOcc together, you'll see that they capture the two core bottlenecks of a physically intelligent landing.
One is the conversion of semantics to space. DA-Nav maps language instructions such as the right-hand turn to the grid position of the image, and AdaOcc maps visual and geometric information to the three-dimensional semantic object. Both are essentially doing the same thing, which is to establish an executable correspondence between semantic space and geometric space of different dimensions.
A common philosophy of the two schemes is to avoid assuming a perfect fit, not to assume that robots will execute instructions perfectly, not to assume that sensors will work perfectly, but to design models using biases, flaws, and inconsistencies as the default prerequisites. This design idea is lacking in previous navigation and perceptual research.
From an engineering point of view, the two works also show a trend in the development of robots in the big model era. DA-Nav is based on the Qwen 2.5-VL visual language model and AdaOcc is based on a pre-trained visual encoder plus a custom decoder. They do not train the entire system from scratch, but take full advantage of the generalization capabilities of the pre-trained model to do task-specific adaptations on this basis. This strategy, which stands on the shoulders of giants, significantly reduces the difficulty of moving from simulation to reality.
Of course, there are obvious limitations to both. Currently, DA-Nav relies on commercial navigation software to provide instructions, and it is difficult to work in areas where GPS signals are poor or there are no maps at all; the reasoning for the model is still on remote GPU servers, and the computational bottlenecks deployed at the end have not been solved. AdaOcc reconstruction of small and thin objects is not precise enough, and performance will decrease in situations of severe overshadowing or geometrical input noise. These are hard bones that require further research.
The value of these two works is that they each provide a set of engineerable technical pathways. DA-Nav demonstrates that robots can complete city-level navigation without high-precision maps using only rough navigation instructions and visual reasoning; AdaOcc demonstrates that three-dimensional occupancy prediction models can achieve sensor adaptation and budget adaptation, and truly serve as heterogeneous robot platforms.
For practitioners who are trying to get robots out of the lab and into the real world, this shift from unable to pump to unusable pump is perhaps more relevant than a breakthrough in a single indicator.
A robot that really understands the real world is getting closer and closer to us.