Why is a large language model scalable? An important basis is that knowledge from a variety of sources is unified into the same input-output space. Input is text, output is text, and the mainstream model learns, combines and migrates knowledge in this unified space.
Today's physical intelligence, perception, movement and world changes often take different forms: from input images to the movement representations of different entities, and then to world changes, and back and forth in different characteristic spaces. Different camera parameters also have different mapping relationships in different entities. The movement of the same cup, changing the camera angle, the pixel changes in different places; the same pixel moves, not necessarily corresponding to the same physical distance.
A 3D-Native World Action Model for Embodied Manipulation (UNITAS) provides the answer: the 3D physical world itself. It describes what robots see, how they move, and how the world changes as a result, unified into the same 3D physical representation space, using the position of points and the trajectory of changes over time to describe observation, movement, and environmental responses in a unified 3D input and output space.
Just as text provides a common input and output space for large models, UNITAS attempts to make Physical AI's input and output grow in a unified three-dimensional physical space.
This is not just a technical idea, UNITAS has demonstrated its superior strength in multiple robot operation assessments: with just 1.7B (1.7 billion) parameters, it achieved an average success rate of 99.8% on LIBERO, 89.1% on LIBERO-Plus, and 92.94% and 93.04% in both Clean and Randomized settings operated by RoboTwin 2.0, respectively, with superparameters of 6B (6 billion), Fast-WAM (8 billion), Motus (8 billion), and π0.5, LingBot-VA, WorldPoint and others taking the real crown in multiple benchmarks, further highlighting the same success rate in full-body robot operations, with an average success rate of 85%!
Address of the paper:https://arxiv.org/abs/2610.12099
Address of the project:
Open source code:https://github.com/DexForce/UNITAS
One, the same model of the world, what has 3D changed?
Seeing 3D, you might be wondering, there are many ways in which the robotics world has introduced 3D information. What does UNITAS' 3D-Native actually mean?
The key difference is that 3D exists either as an auxiliary information or as the basis for understanding the world as a model.
Some of the previous methods could be called 3D-aware World Action Models. They introduce depth, dot clouds, or geometric constraints to help the model obtain additional spatial information, but the core of the model is still based on two-dimensional vision or other potential characteristics. In other words, the model is capable of sensing and sensing three-dimensional information, but the three-dimensional is not the native language of the world it understands.
For example, the video world model predicts the future state primarily through continuous images. It can judge how objects in the image change, but the real distance, spatial relationships, and laws of motion between objects require models to further infer from two-dimensional visual changes.
Part of the 3D method further enhances the ability to understand space. PointWorld, for example, is able to predict three-dimensional scene changes, but the trajectory of action is input from outside, and the model is closer to a simulator that predicts environmental change, rather than at the same time understanding the unified model of why action is happening and how the post-action world is changing.
DreamZero, including LingBot-VA from Jingle Wave, and DreamZero from Ingrid, developed mainly along the path of two-dimensional visual modeling, combining future video prediction with robotic motion generation. This path was able to leverage the rich experience in video, but the main driver of world change was still the two-dimensional image and its potential characteristics, the distance between objects, the location and the relationship between motion, from which models needed to learn and infer.
The core change in the 3D thinking explored by UNITAS is that it no longer allows models to infer three-dimensional worlds from two-dimensional vision, but instead directly allows models to learn the world in three-dimensional physical space, in short, it works directly on a three-dimensional Token under a unified coordinate system, rather than through an image Token as a bridge.
The main UNITAS model deals directly with three-dimensional tokens and uses three-dimensional space as a common representation of robots observing, acting and understanding environmental changes:
Observation: visual information corresponds to three-dimensional positions;
Action: the trajectory of three-dimensional points on the operating body that change over time;
Scenic Dynamics: The motion trajectory of the observed points in the same time space.
When the cup is picked up, how close the claw approaches, how far the cup leaves the table, and how far it eventually moves, can be expressed in the same coordinate system with real distance measurements. The model can both generate executable movements based on the task, and can predict the scenic changes that a given move will bring, connecting robotic controls to world predictions.
According to the paper's authors, UNITAS is the world's first native 3D world model for robotic missions. UNITAS is exploring a way for robots to understand the physical laws of the world from the perspective of observing the world through images.
UNITAS overview. (a) Multibody data: simulation, virtual machines, human hands; (b) a world coordinate interface: movement flow and scene flow modeled together in the same coordinate system; (c) leading on multiple benchmarks with 1.7B parameters.
Second, how robots move and how the world moves are the same questions.
The common strategic model is learning "how should I act"; the traditional world model is learning "what will the world be like next". This is seen as two heterogeneous problems, optimized with two sets of goals respectively.
However, from the point of view of the physical world, the fact that the robot reaches out and moves the cup and the cup is not two problems, it is an overall and causal event.
UNITAS describes the process using three-dimensional trajectories:
Action Flow: How the dots on the claw of a human hand or a robot move through time.
Scene Flow: These points are conditioned on how the points in scenes such as cups move.
In the eyes of the model, under the conditions of mission instructions, predicting how all the relevant points in the physical world will move next. The dissolution experiment of the paper found that, after Scene Flow monitoring, the performance of the action strategy itself continues to improve. It helps robots "understand what is happening in the world" and "know better what to do".
Predicted action flows and corresponding scenarios
In order to achieve such a joint modeling and to jointly study on a wide variety of objects and datasets, it is first necessary to solve the problem of the shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow shallow.
1. UNITAS proposes embedding World-Aligned 3D Position Embedding modules into a unified three-dimensional coordinate encoding for observation, action, and scenic flows; for depth gap data, the model can learn full depth in the field of vision to support the training and reasoning of RGB-D and RGB data.
2. For the case of trajectory lengths (Long Horizon vs. Short Horizon) and frequency differences (10Hz vs. 30Hz) in different datasets, UNITAS has proposed the Physical Time Trajectory Tokenizer module, which encodes the trajectory of different parameters in a point-to-point manner, using FiLM modulation to encode a trajectory into a unified token to align different trajectories with the same physical time characteristic space in the mainstream model.
After solving the above data misconceptions, the main UNITAS model Point Dynamics DiT can be studied in a unified characteristic space: Observation: three-dimensional points under the same space; Action Flow: sampling point trajectory Token + initial point location coding under the same space; Scene Flow: scenic trajectory Token + initial point location coding under the same space). UNITAS also has robotic control and scenario prediction capabilities: strategy patterns can execute actions based on task generation; simulator patterns can change scenarios based on a given action trajectory.
Three, human hands, simulations, real-machine trajectories, in the eyes of the model, are the same fluff of language.
The human hand holds the cup, the robot handles the cup with its claw, and the physical structure and controls are different. However, if the focus is on the movement itself, both can be used to describe the approach, grasping and moving process using points on the operating end.
The clawed claws in the simulation, the clawed claws of real robots, and the human hands, although from different data sources, are all expressed as three-dimensional motion trajectories. For UNITAS, they are the same clawed language claws.
Shared point flows across data sources: simulation, real robots, human hands, are all point trajectories within the same coordinate system within UNITAS; shown in Figure 3 on the right shows mixed data training continuously reducing error at the end-user (source: UNITAS paper).
UNITAS uses these three-dimensional point trajectories to represent that human hands, different robots, and operational data in simulation can participate together in training. Models can thus learn interaction processes such as approach, grab, and move from different sources of experience.
The paper uses nine data sources for mixed pre-training, then tweaks to specific tasks, and observes improvements in operational performance. On LIBERO-Plus, mixed pre-training increases overall success rates by 2.6 percentage points; of these, the success rate under robot initial state disruption has increased from 84.8% to 94.2%.
Four, with 1.7B parameters, the 8B model wins! UNITAS leads the world in multiple ratings
In the field of physical intelligence, the competition for model capabilities will ultimately return to comparable ratings. UNITAS won the overall victory at LIBERO, LIBERO-Plus, RoboTwin 2.0 and VLABench with 1.7 billion parameters:
An average success rate of 99.8% for LIBERO reached SOTA. The long-range mission subset LIBERO-Long also reached 99.4% and maintained a high success rate for tasks requiring continuous multi-step operations.
92.9% / 93.0% RoboTwin 2.0 two-armed operation success rate
89.1% LIBERO-Plus overall success rate. In the face of seven types of disruption, such as camera angle, robot initial state, the model still maintains a strong ability to complete tasks. In these, the robot initial state disruption success rate is 94.2%, demonstrating its ability to adapt to changes in physical conditions.
In the RoboTwin simulation test, the average orbital error at the point of motion decreased from 0.656 cm at PointWorld to 0.427 cm, with a prediction accuracy rate of no more than 1 cm reaching 92.31%, all four scenarios leading the way.
RoboTwin action conditions scenario forecast: UNITAS leads on all four indicators, moving ADE from 0.656 cm to 0.427 cm
Real robot testing further tests UNITAS's execution effects in real physical environments.
In the actual assignments reported in the paper, the average success rate for UNITAS was 85%, with 90% success rate for Organize Books and 80% success rate for Pack the Ball.
Overall, UNITAS's performance is not a breakthrough in a single indicator, but reflects its comprehensive strength in many aspects such as standard operations, two-arm collaboration, disruptive adaptation, scenic motion prediction, and real robot execution.
More remarkably, UNITAS has only 1.7B parameters, but it outperforms the larger parameter-scale comparison model in many indicators and performances. This shows that the value of UNITAS is not only confined to the technical concept of 3D nanotechnology, but that it has brought the ability to operate robots into a real physical environment, providing a truly efficient path for robots to learn from multiple sources.
UNITAS with a small victory key: dissolution experiments verification design by design
Removing all of the original 3D designs, the model degenerated into a conventional VLA baseline, and the overall success rate dropped from 86.47% to 67.58% with a gap of nearly 19 percentage points, all contributed by the expression itself.
The success rate of disrupting the camera's perspective after the 3D location coding was removed dropped from 79.11% to 66.73%, and the alignment of 3D information is the key to widespread cross-visual communication.
The overall success rate decreased from 85.74% to 78.89% following the removal of the chip, and aligned input observation and output action spaces are central to the study of generalizable robot action controls.
The success rate plummeted by 19.87 percentage points following the removal of the historical token shell after the robot's initial state was disrupted.
The overall success rate on all three topics increased from 85.74% to 86.47, while "understanding how the world works" in turn made "how to move" more accurate.
The four components share their functions and each other's advantages: each point saved by the parameters is saved by a way that is more appropriate to the physical world.
Five, Task Anchor: Install long-range task capabilities to the base model
In addition to understanding spatial relationships, robots need to know what they've done to execute tasks.
For example, if a mouse puts the red block on a blue hook and then puts it back in its original position, in this task, the red block is also stuck on top of the hook, the next step is to put it down or put it back in its original position.
Another study on cross-dimensional intelligence, TaskAnchor ("Grounding Task State in Reactive VLAs for Long-Horizon Manipulation"), focuses on this blurry task state confusion. It extracts context from the execution history by inputting language to tell the model at which stage the task was performed, while adding historical clues to visual characteristics to help the visual language action model (VLA) select the next action.
This lightweight module provides access to different base models without changing the original motion generation mechanism. With π0.5 access, TaskAnchor increased the average success rate from 14% on RMBench to 79%, and increased the cumulative success rate from 38.7% on RoboMemArena to 64.7%, with an increased parameter of about 27 million, with an additional delay of about 2 milliseconds for single strategy calls.
In the Human-Cued Matching real-machine mission, humans first pick up and drop an object, and robots then, according to previous clues, grab the corresponding targets. It demonstrates the ability to use historical context-adjusted behavior to complement UNITAS' models of spatial interaction.
Human-Cued Matching (a1-a3) Human-Cued Matching: A human gives clues with a demonstration of action, and the robot matches the target and completes the operation accordingly. (b) Raises the red block three times in a row.
Address of the project:
Two papers responded from different perspectives to how robots could better use experience: UNITAS gave observation, action and environmental change a common three-dimensional language interface; Task Anchor helped robots combine previous experiences to determine where the current task was taken and what to do next. The team said that the future plans would integrate Task Anchor's related capabilities into UNITAS, allowing the model to understand spatial interaction while also using historical contexts to accomplish longer tasks.
Unified representation × generated simulation: Physical AI's data flywheel
UNITAS's exploration of 3D native world action model routes has not only changed the way robots learn and understand the physical world, but has further expanded the value of each link in the Physical AI technology chain.
In the past, robotic learning relied heavily on data collected from real environments, but the cost of obtaining data from complex physical scenarios was high and the coverage was limited. When models began to be unified in three-dimensional physical space, the form of experience required by robots changed as well: the location of objects, trajectories of movement and interaction processes became an important vehicle for describing and transmitting physical experience, no longer just training data.
In this trend, the role of simulation has expanded further. It is no longer just a data supplement, but an important link in connecting data, models, and real applications. The cross-dimensional DexVerse® simulation engine can generate three-dimensional scenes based on inputs such as language, images, combined with physical simulation and sensor simulation, and produce synthetic data. EmbodiChain combines scene generation, data expansion, and model training, allowing robots to accumulate experience in a variety of virtual environments by changing the properties of objects, scene layout, and observation conditions.
This is in line with the training requirements of UNITAS. Real cameras are difficult to directly record where the clock goes at the same point, but simulations can calculate continuous trajectories based on the state of the object and provide monitoring for the model's learning of movement and environmental response. The three-dimensional motion process generated by the simulation is the basic language of UNITAS learning physical interaction.
To this end, the cross-dimensional Physical AI data fleet has been formed that generates simulation of tokens, models of tokens, learning of tokens, authentication of tokens, data reproduction, addiction to tokens, promotes experience across viewpoints, cross-bodies, cross-scenario migration, and explores the core direction that Physical Token economics is focused on: continuously reducing the cost of robots gaining the next capability.
The paper also demonstrates the potential in this direction: UNITAS's scenic flow monitoring is based on simulation only, but realistic observations of DROID, RH20T, given the trajectory of the claw movement, can still reasonably predict scenic changes.
Zero-sample migration: UNITAS directly predicts the flow of scenarios on real-world imagery. While the X-ray model has not seen 3D monitoring of these real-world scenarios, it can still predict "how the world will change after action" (source: UNITAS paper Figure 11)
The big model has found a unified language in the world of knowledge, but Physical AI has not yet. This is an inevitable dilemma with physical intelligence: without unified representation, data is difficult to assemble; data is difficult to assemble, and scaling is never talked about.
As for the concept of the world-modeling puzzle, anyone can talk about it today. More interestingly, the native of Physical AI suggests what space should be like.
The language model has been proven: finding a unified representation will result in scaling.
Now it's time for the physical world.
Papers and projects:
UNITAS: A 3D-Native World Action Model for Embodied Manipulation
UNITAS project page:
TaskAnchor: Grounding Task State in Reactive VLAs for Long-Horizon Manipulation
The project page for Task Anchor: