Finally, robots not only know where to go, but also know where to stop, in what direction to face the target, and even after stopping, they can work directly.
It sounds natural, but for robots, this is exactly a problem that has been around for a long time.
For example, allowing a robot to pick up a cup on the table. Being able to walk to the table does not mean being able to pick up the cup. The bottom of the plate stops a little far, the mechanical arm may not be enough; towards a slight deviation, subsequent crawling may also be affected.
How can robots not only walk, but also stop, pick up, or even perform more sophisticated operations?
Recently, the national team has given an interesting new route.
Together with the CITLab of Physics and Intelligent Technology at the University of Cincinnati, Fujitsu AI has proposed the unified world-action model Fysiverse-UniWAM, which attempts to combine autonomous navigation, mobile operations and the relatively independent capabilities of the three types of workplace agility.
It is also worth noting that the model does not follow the traditional route of mass robot pre-training as a starting point.
Starting with a pre-trained video world model, the team combined navigation and operation learning with a shared backbone network and proposed MAP (Manipulation Anchor Pose, Operating Anchor Positioning Positions) to construct more than 1.5 million tracks, totaling about 7,500 hours of MAP-Data data.
It's the same technology, but how does it actually work?
The result was a bit surprising.
In the MAP-Bench benchmark test, the average position error for Physiverse-UniWAM was only 9.85 cm and the average directional error was 2.30°, with the strongest external baselines decreasing by 30.1% and 44.0%, respectively.
Furthermore, the team directly deployed the model to four robotic platforms and conducted 24 real-machine tests covering autonomous navigation, mobile operations and fixed workplace operations.
Of these, 13 navigation missions led the way in positioning errors, and four mobile operations were first in success rates; overall performance in complex fixed-site operations was also close to π0.5 after large-scale robot pre-training.
How can a model achieve such a feat without extensive robot training?
24 real-machine tests, how much can a model do?
I'm not in a hurry to look at the technicalities and go straight to the real thing.
Autonomous navigation: Not only can you walk, but you can also stop
The researchers moved the robot to a position far from the target, changing the starting point and direction, and the robot could approach the route according to the current environment's autonomous planning, adjusting the position and direction of the dashboard, and ultimately accurately stop.
Even when changing targets, scenarios, and even robotic platforms, the model is still able to complete the corresponding navigation tasks.
It's not just a demonstration effect.
In 13 MAP navigation simulator tests, UniWAM's position errors all ranked first, with the best result towards errors in 11 of those missions.
The purpose of precision suspension is to create the right conditions for the next operation of the mechanical arm.
Mobile operations: Get to the point and do it live
In mobile operations, robots need to approach targets autonomously, adjust the chassis, and then control the mechanical arm to complete the grabbing and placement.
For example, from moving to the side of the table to reaching for the cup and receiving, the whole process is done by the model itself.
Instead of classifying the colored squares, robots need to identify the target colors, coordinate the chassis with the mechanical arm, and place the squares in the corresponding tray.
The real test of these missions is their ability to navigate and operate, not just to complete a single action.
Technical Report Table 7 gives individual assessments of 4 real-machine mobile operation missions. UniWAM-Scaled has the highest success rate of any of the 4 mobile operation missions, with an average success rate of 80.0%, leading the current strongest robot pre-training baseline by π0.5 to 17.5 percentage points.
Mission-by-mission evaluation of 4 mobile operations in Table 7 of the Technical Report
Easy to use: grab everything, three-layered cups with two arms, high-precision syringe, long coffee making
If navigation and mobile operations have already demonstrated the integrated capabilities of the model, then the next agile operation will be even more promising.
The first is a "Pick Up Anything" interactive presentation.
Faced with objects of various shapes, sizes and postures, robots are able to interact with different targets in a continuous manner based on real-time visual observation.
When the human hand moves to different positions in three-dimensional space, the mechanical arm continues to perceive the position of the hand, and accordingly adjusts the delivery movement, completing the precise delivery.
More difficult operations are also not mentioned.
In a three-tiered cup-laying mission with two arms, the robot needs to coordinate the two arms, accurately grab and place them in layers, and eventually build a three-tiered cup tower.
In high-precision test tube insertion tasks, robots need to control the position and posture of the end of the mechanical arm to complete precise alignment and insertion, posing higher requirements for spatial precision.
In long-range coffee production tasks, robots also need to complete multiple operational steps in succession, posing greater challenges to long-range task planning, motion welding and execution stability.
From widespread grab-and-arm collaboration to precise insertion, Physiverse-UniWAM demonstrates a wealth of sophisticated operations.
Of the 7 fixed-to-station machine assessments, the model had 3 tasks above π0.5, with an overall success rate of 68.9%, only falling behind π0.5 by 2.5 percentage points.
Success ratio for fixed-site operations of tasks in Table 7 of the Technical Report
Technical Report Table 6 summarizes the overall performance of the three categories of missions: UniWAM-Scaled achieved the best accuracy of anchorage for MAP navigation and mobile operations, ranked first in mobile operations success rates, and second in fixed-site operations success rates.
Table 6 of the Technical Report shows the overall assessment of 24 real-world missions.
What is the technical secret behind a unified model that has not experienced massive robot pre-training and is able to lead not only in navigation and mobile operations, but also to accommodate such a wealth of agility?
What path has UniWAM taken without extensive robot training?
In recent years, large-scale robot pre-training, such as the π0.5 GR00T, has become an important technological route for physical intelligence.
But real robot data is not easy to obtain. In particular, the complete mobile operation trajectory, which requires continuous collection of the entire process from mobile, stop-and-go to the execution of tasks with mechanical arms, is not low in cost and complexity.
More importantly, navigation differs significantly from operation itself: the former focuses on environmental layout and bottom movement, while the latter focuses on close object interaction and precise control of the mechanical arm.
So how can we not rely on a mass of complete robotic trajectories to learn navigation and operations together in one model?
For this reason, the team split the problem into two levels.
The first question is: How do robots know where the navigation terminal is suitable for operation?
The second question is, how do you get the data to navigate and operate differently, how do you get them to learn together in a model?
The answer is MAP and Mixed-Stream.
Challenges faced by mobile operations and solutions to Fysiverse-UniWAM.
MAP: Let the end of navigation be the beginning of operations
Traditional semantic navigation typically requires robots to only reach near their targets, but for mobile operations, reaching targets is far from enough.
The robot also needs to know where the substrate should be parked, in which direction to face the target, in order to facilitate the next movement of the machine arm.
For this purpose, the team proposed a MAP (operational positioning positioning), which would be the reference position and orientation for subsequent operations, as a clear navigation learning goal.
More importantly, MAP supervised data can be generated automatically on a large scale.
Using HM3D, InteriorGS, and private large-scale scene data, the team combined scene geometry information, visual language models, and automatic path mapping to generate reference landing positions and approach trajectories for different operational targets.
Each trajectory includes not only visual observation and navigation movements, but also recorded geometric monitoring information such as target boundary frames, MAP image projections.
MAP-Data is an automated process for generating data.
The resulting MAP-Data consists of more than 1.5 million tracks, approximately 7,500 hours of data.
This means that robots don't have to rely entirely on real mobile operating models, but they can learn how to get to the right place by generating data on a large scale.
But solving a problem where a mother is parked doesn't mean solving how to learn a mother together.
Mixed-Stream: Learning to navigate and operate separately, sharing model skills
There are significant differences between the visual information required for navigation and operation, the operating space and the control method.
Directly splicing the two types of actions together may increase the difficulty of unified learning; training the two models, respectively, makes it difficult to fully share skills.
Fysiverse-UniWAM takes a different approach: two independent data streams, shared by a core backbone network.
Fysiverse-UniWAM Mixed-Stream overall architecture.
Navigation flows specialize in environmental observation and bottom movements, and operational flows specialize in multi-point observation and mechanical arm movements.
The two streams of data have their own input encoders and action output heads, but share a unified video DiT and action DiT backbone network to achieve synchronous modeling of visual dynamics and action generation through the Mixture-of-Transformers (MoT) mechanism.
More importantly, navigation data and operational data do not have to come from the same complete mobile operational trajectory.
In training, the two types of data can be independently sampled, batch input, and jointly update the shared skeleton; in reasoning, the two data streams can both work independently and execute simultaneously.
This allows mass navigation data and fixed-site operations data to participate in unified model training together, reducing reliance on complete mobile operations demonstration data.
In addition, the model also introduced image plane-assisted monitoring: navigation flows predict target boundary frames and MAP projection locations, operational flows predict two-dimensional projection trajectories at the end of the mechanical arm, and further enhance the link between visual information and motion prediction.
Navigation MAP prediction and visualization of end-of-operation trajectory prediction
In this regard, the technical logic of UniWAM is clear:
MAP solves where robots should stop, Mixed-Stream solves how to learn to navigate and operate together.
The combination of the two allows for relatively independent navigation, mobile operations, and fixed workplace operations to be modeled together in a unified world-moving model.
How effective is this route to move from benchmark testing to zero samples?
To validate this design, the team constructed a MAP-Bench containing 15000 trajectory ratings against models such as π0.5, GR00T N1.7, FastWAM, Uni-NaVid and InternVLA-N1.
The results showed that UniWAM-Scaled achieved the lowest stopping errors in short, medium and long-range missions, with an overall average position error of only 9.85 cm and a directional error of only 2.30°.
Compared to the strongest external baseline, the position error decreased by 30.1% and the error direction decreased by 44.0%.
Table 4 of the Technical Report shows the performance comparisons between the MAP-Bench models.
Furthermore, the advantages are not entirely derived from additional data.
Unless MAP-Data is introduced, UniWAM-Base has outperformed the external baseline; with the addition of MAP-Data, suspension accuracy is further improved.
Architectural control experiments have also shown that Mixed-Stream can improve success rates and accuracy of mobile operations compared to Unified Action Token, Independent Action Experts, and Sequence Splicing.
What's even more interesting is the migration of zero-sample machines.
The team used only MAP-Data to train UniWAM-Sim, not including real robot demonstration data, and then deployed it directly to two real robot platforms.
Eventually, the model achieved a position error of 18.7 cm and an orientation error of 6.34°, demonstrating the potential to migrate from generated scenario data to real robots.
Technical Report Table 8 shows the results of the migration of the UniWAM-Sim Zero Sample Virtual Machine
This further demonstrates that the improvement of robot capabilities through clear geometric oversight and unified model design cannot depend solely on expanding the actual scale of robot pre-training.
Unifying robotics capabilities, not just expanding the pre-training scale
Fysiverse-UniWAM is truly worthy of attention, not only in terms of navigation and mobile operations indicators.
More importantly, it demonstrates a different robot's ability to build paths.
Through MAP, navigation is truly directed to follow-up operations; through Mixed-Stream, decentralized navigation and operational data are taught together in a world-moving model.
This does not mean that large-scale robot pre-training is no longer important, but rather that it is not necessarily the only prerequisite for building a unified robot capability.
From precision navigation to mobile grappling, to two-arm collaboration and precision operation, Physiverse-UniWAM has demonstrated the breadth of a unified model's capabilities with 24 real-time machine tests, demonstrating the potential of this technological route.
When physical intelligence is no longer just the size of robot data, how to really integrate different capabilities may also be the key to a universal robot.
Finally, the accompanying projects and open source resources:
Project homepage and presentation
Technical reports
GitHub code
Hugging Face Navigation Weight
MAP-Data is a data set
MAP-Bench data and evaluation trajectories
© THE