What is our ideal model of physical intelligence?
Perhaps it can be summed up in a few key words: genericized enough to understand the real world, capable of independent decision-making and action, and able to interact naturally and efficiently with people and the environment continuously.
These keywords together point to a class of models: the World Action Model (WAM). The video-generation model is becoming the backbone of the robotic world model, and it is extremely important to understand the depth of the real physical world if you want to really scale up a physically intelligent model.
This is a new development in a model of physical world action, where mechanics are able to perform relatively complex and subtle real-world tasks with stability:
Recently, Chevrolet Intel released its Intelligent Pre-Training Model Noe-0, a world-action model that trains the entire chain using non-core data collection.
The link to the tech blog:
All of the self-training data comes from records of people performing operations directly in real environments -- over 50 cities, hundreds of thousands of hours, hundreds of thousands of task types.
This is a fairly radical strategy. Noe-0 demonstrates the ability to complete pre-training and post-training models of the physical world without relying entirely on traditional remote control data, and to develop the ability to perform real tasks on real robots.
Noe-0: Full-Link "Real Operations" training body model
What is real "real operation"?
Traditional remote-controlled data often incorporates a large number of standardized workstations in a concentrated location: similar desktops, finite objects, and pre-designed task processes. Although such environments can also be called "real scenarios", they are essentially man-made for data collection.
In other words, data factories can replicate jobs, but it's hard to replicate the real world itself.
Therefore, in order for robots to learn about the real world, it is necessary to get rid of the robotic body and fixed jobs. Data collection actually enters everyday environments such as homes, shops, collectors directly face objects and tasks in real space, data can naturally contain different perspectives, lighting, layout, operating habits and a large number of difficult to predict long-term situations. Compared to standardized workplaces, this data is closer to the real world that robots will face in the future, and can better reflect the complexity and diversity of the real world.
In addition, the research team observed that the same task requires 3 to 5 times more time to complete the same task by remote operation than is usually required for direct operation, which not only increases the cost of collection, but also makes it easy to generate slow, mechanical motion data.
In order to achieve real-world data acquisition, the company has created an efficient, natural, and optimized data acquisition device called RoboPocket (RP).
The team found that using the claw shapes directly along the robot-facing design would limit the range of operations of the collector in the real environment. For this reason, RP combines the claw shapes, end shapes, sensors, and collection methods to emphasize "homogeneous design": the collection end is consistent with the robot's end as much as possible while respecting the user's operational comfort and collection efficiency.
RoboPocket (RP) designs
The entire link has no physical data.
For Noe-0, non-core data, "real operation" data, is the data base for both pre-training and post-training.
It is well known that a large model's training chain is longer, and both pre-training and post-training have a huge impact on the final capability of the model. It is noteworthy that NOE-0 uses non-core data in both pre-training and post-training stages.
In the pre-training phase, Noe-0 uses large-scale distributed no-body data to allow the model to see as many scenarios, objects and tasks as possible, from which it learns visual changes across tasks, state transfers and rules of action, and builds basic real-world capabilities. After entering the post-training phase, the team collects more focused no-body data around specific tasks for task adaptation and capability enhancement.
More importantly, this data paradigm runs through the entire training chain.
If the pre-training uses non-material data, and the post-training retracts the remote-operating data, there will be a breakdown of the data distribution and the pattern of movement between the two stages: the model learns the natural laws of operation in the pre-training, and needs to re-adapt a set of non-matched motion expressions in the post-training.
Noe-0 always follows the same non-core data route from pre-training to post-training, keeping the learning of basic skills and the adaptation to specific tasks continuous and avoiding the bottleneck of re-training as a whole link.
World Action Model: Transforming the real world experience
When data really got out of the standardized workplace and into the real world, the abundance of data also brought new challenges, and traditional VLA technology routes were no longer able to meet demand due to their limited generalization capabilities. Different environments, objects, lighting, perspectives, and more complex and continuous interaction processes required models with a stronger understanding of diversity in video and world modeling capabilities.
This is why the more real-world data is available, the more urgent the need for a new generation of the World Action Model.
In the recent Robotics.