The main page of the project:https://loop-wam.github.io
What is the meaning of this article?
Open source code:https://github.com/loop-wam/LoopWAM
Recently, byChinese University, Nuclear Power Machines and the Late Bronze AgeLoopWAM, a co-developed world-action model, topped the RoboChallenge Table30 V2 ratings, topped the process-to-process double-digit rankings, and outperformed similar models such as VLA-DM0.5. LoopWAM also achieved the same SOTA performance on the simulation lists of RoboTwin 2.0 and RoboCasa 365.
LoopWAM's models used in all list evaluations are based on:The official Karachi-based PPU-810EIt's training.Other than NVIDIAA series of cards.
Behind this achievement is a technological path that continues to learn from real deployments: allowing success, sub-success, and failure experiences to expand the learning scope of the world model together, enhancing understanding of mission processes and the consequences of action, and improving action generation through quality guidance to enable robots to do better in the next deployment.
Starting with the initial model using only expert demonstration training, after three rounds of deployment and training, the overall success rate of LoopWAM improved by about 19.7 percentage points, a relative improvement.About 83%The process score has also been raised from 36.10 to56.80。
Making real deployment experiences the starting point for the next round of progress is key to LoopWAM's rise to the top.
This idea, which evolves continuously from real-world deployment feedback, is consistent with the PI API workflow scheme recently released by Physical Intelligence: improving the cyclical advancement capabilities of deployment robots by micro-adjusting the deployment of mouse models to feed data. LoopWAM goes further in the use of deployment data: the closed-loop update does not require additional expert demonstrations, nor the introduction of reinforced learning, but instead takes advantage of the features of world models learned from action and environmental change, incorporating all of the successful, secondary, and failed rollout trajectories into video exercises, then improving the quality of production through quality steering and weighted action supervision, so that the deployment experience itself becomes a source of continuous improvement.
This is the first time that I've been able to do this.
01
30 integrated capabilities for verifying real-machine missions
RoboChallenge Table 30 V2 covers the four robotic entities of Aloha, DOS-W1, ARX5 and UR5, and includes a variety of tasks such as organizing objects, using tools, fine-tuning operations, and arm synchronization.
Figure 2 RoboChallenge is a multi-tasking visualization
Challenges in real operations are present in every interaction with the movement: whether to select the target, whether to grab it steadily, whether to use the tools in place, or whether to complete multiple operational steps in succession. Table30 V2 measures the completion of the task and the quality of the execution process by dividing the success rate and the process, respectively.
In this assessment, LoopWAM achieved success rates on 11 missions.60%The nine mission processes scored 80 points and above. Several groups of specific results made the model's capabilities more intuitive:
Organizing fruits, sorting items, extracting green blocks: success rates are100%。
Using small objects: success rate90%It's a process.92。
Press the button: Success rate80%It's a process.92.5。
Stacking bowlware: success rate80%It's a process.89。
From identifying targets, completing classifications, to operating tools, advancing multi-phase tasks, high scores are distributed across different entities and task types, all supporting the overall performance of LoopWAM.
02
Getting Success and Failure into a Closed Learning Circle
Behind the top, LoopWAM focuses on a question that directly affects the continued progress of robots: how to use the experience gained in real deployments?
Expert demonstrations provide a good starting point for the robot to learn. But once it enters the real world, the robot encounters an out-of-the-ordinary state: picking up diversions, accidental slippage of objects, insufficient contact with tools, or only part of the task being completed.
These deployment tracks contain a wealth of physical interaction information as well as a mix of qualitatively different movements. All movements are treated as imitating targets, potentially reinforcing inappropriate behaviour; only success tracks are retained and a great deal of experience about the movements and their consequences is lost.
LoopWAM has therefore proposed a closed-loop world action model: incorporating success, success and failure experiences into unified training, enhancing world models with broader real-world interactions, and leveraging quality guidance and action oversight to keep the strategy oriented towards high-quality behaviour while absorbing experience.
Figure 3 LoopWAM closed loop self-elevated
The full name of this work is LoopWAM: Closed-Loop Generalist Policy Improvement from Deployment Rollouts.
Deployment → Collecting successes, trajectories of success and failure → Quality perception training → Re-deployment
Each new round of experience is trained in conjunction with expert demonstrations and previously accumulated trajectories, forming a closed loop of continuous iteration.
03
Key technologies supporting the self-improvement of the closed loop
LoopWAM revolves around one goal: to take full advantage of the various experiences in real deployments, to enhance the world model's understanding of tasks and physical interactions, and to transform this understanding into higher-quality action. For this, it combines data distribution expansion, action-condition world modeling, video and action co-generation, and quality perception learning. LoopWAM builds video and action bi-branch architectures based on the Wan2.2-TI2V-5B pre-trained video model, and jointly predicts future images and robotic action within the same timeframe.
Figure 4 LoopWAM method
Embracing more real deployment data, expanding the scope of learning for world models
Expert demonstrations provide quality paths for completing tasks, while real deployments lead to a broader range of states, actions and outcomes. LoopWAM incorporates successful, sub-successful and failed trajectories into training, retaining video predictive supervision of all quality trajectories. More extensive interaction processes thus become the learning material for world models, expanding their ability to represent environmental changes and physical interactions.
Strengthen ACWM to help models understand the consequences of their actions
Action-Conditioned World Modeling (ACWM): uses actions already performed to predict their corresponding visual outcomes. The key is the source-related noise sampling, which controls video and action noise levels in training respectively.
For expert demonstration, the model recovers the corresponding future images from the loudest noise, while learning the motion generation, allowing the model to learn how to complete the task. For deployment trajectories, applying less noise to the action, retaining more motion information, allows the video branch to change the environment caused by the learning action. Even if the task is not completed, the link between the action and the result can still be used for training, promoting the consistency of the predicted video and the generated action in behavior and results.
Let the world model directly participate in action generation
In the same combined de-noise process, video and motion information continue to interact through two-way attention at each layer. Rich video prediction supervision helps the model learn better about the physical world and the consequences of motion, both of which together support motion generation: the model both learns how to best accomplish the task of making noise, and learns how motion generated by making noise can change the environment, allowing the world model to more fully support robotic control.
Distinguishing data quality and guiding high-quality action
In order to make it clear that the model is behaving better while absorbing more experience, LoopWAM assigns quality scores to the trajectory and adds quality conditions to the task instruction, driving the video and action branches by way of quality steering. During training, the model learns the distribution of different quality behaviours under the same task; in real deployment, specifying expert quality conditions will generate guides to high quality behaviour distribution.
In addition to quality conditions, LoopWAM also has the authority to monitor movements based on trajectory quality adjustments. While maintaining full video predictive monitoring on all trajectories:
The expert trajectory provides comprehensive monitoring of the movements.
Subtractive trajectories are weighted by mass, reducing the intensity of imitation of poorer movements.
The direct action oversight weight of the failed trajectory is set to zero, and the real consequences of its interaction are still used to model the world.
Quality steering is responsible for distinguishing between different behavioral distributions, overseeing weighting to determine the extent to which different types of actions are imitated. The two work together to allow models to learn the world from broader experiences and guide actions to generate biased high-quality behaviors that support real-world deployment.
Figure 5 Video predictions of the four missions
The motion above shows the video prediction effect on the four tasks of sorting fruit, picking pens, stacking bowlware and scrolling dust removal. The left side of each group is the actual observation when the robot is executing it. The right side is the model prediction. The controls are visible, the prediction video can reflect the movement of the mechanical arm, the movement and operation of objects, and the intuition shows the consistency between the video and the motion prediction in the main movement and operation processes.
This consistency is facilitated by training and generation mechanisms: when training, rich interactive data provides a correlation between action and actual outcomes, source-related noise sampling strengthens ACWM, enabling models to learn the environmental changes that action will cause; when generating, video and action continuously exchange information through two-way attention in the same combined de-noise, making predictions of the future interconnected with the action being generated.
04
Three rounds of deployment lead to sustained improvement
Whether the closed-loop is effective can be seen directly in the changes in the three-round deployment. In the Table30 V2 experiment, which covered 30 missions, LoopWAM continued to absorb its own real-world deployment experience and improved multitasking performance through three-round closed-loop updates.
Overall success rate: 23.67% → 28.00% → 37.67% → 43.33%
The process is: 36.10 → 42.63 → 51.65 → 56.80.
Figure 6 Multiple missions are upgraded
In the three updates, two overall indicators were improved each round. The results showed that LoopWAM was able to convert its experience of deployment in multi-tasking environments into follow-up strategy improvements.
05
Continued self-improvement in overlapping tasks
In the face of difficult single-task post-training, LoopWAM is also able to take advantage of real deployment experience and continuous improvement. In overlapping tasks, LoopWAM is able to start operating from randomly dropping clothing in front of the robot, cluttered clothing. The initial state of clothing is chaotic and complex.
The robot must first find the right grip position in a chaotic state, by repeatedly grabbing and gradually unfolding, exposing the edges of clothing, and then grabbing the edges of clothing to complete the subsequent folding. Clothing changes as the operation continues, which requires the model to continuously adjust the movements according to the actual changes at each step.
LoopWAM began with a multi-tasking model, using newly collected expert overlay demonstrations for single-tasking post-training, followed by two-wheeled deployment-driven self-improvement.
The success rate of overlapping shirts increased from 13.20% to 32.54%, eventually reaching 55.56%; the process score increased from 14.44 to 67.78.
Figure 7 Training for self-elevation after wearing clothes
Figure 8 Displays the effect of clothing folding (operating part 10 times faster)
This suggests that continuing to use real-world feedback after the model has been adapted to a difficult task can still bring significant gains.
06
Further verification in the simulation benchmark
On two simulation benchmarks, LoopWAM further validates self-improvement in the broader sense: model updates can take advantage of trajectories generated by arbitrarily deployed strategies, even if those strategies perform poorly. The deployment experience from different strategies extends the learning state, action and result distribution, LoopWAM learns world changes through quality perception training, and guides the action to generate biased, high-quality behavior.
LoopWAM achieved an average success rate of 94.5% on RoboTwin 2.0, which contained 50 two-armed missions, with a standard environment of about 94.6% and a domain randomization environment of about 94.4%, all indicators exceeding methods such as LingBot-VA 2.0, achieving SOTA.
Figure 9 Reviews of RoboTwin 2.0
LoopWAM achieved an overall success rate of 49.5% in the 50-mission evaluation of RoboCasa 365, up from 46.6% for ABot-M0.6.[6] The model also achieved a 14.9% success rate for 16 unseen tasks excluding expert training data and deployment training data, demonstrating the ability to migrate to unseen task portfolios.
The results of the RoboCasa 365 evaluation are as follows:
Both sets of experiments did not carry out additional physical data pre-training, but used pre-training video models to learn from expert demonstration and deployment trajectories. Combined with real robot experiments, LoopWAM scored best on both real and simulated three benchmarks, providing sufficient evidence for the utility of mixed-quality deployment experiences.
Let every real deployment bring a learning experience and continuous self-improvement in the real deployment, which is the direction LoopWAM wants to continue to move forward.