RSI (Recursive Self-Improvement, recursive self-improvement) focuses on whether systems can constantly discover their own shortcuts, gain new experiences, and improve the next round of performance with new experiences. However, current embedded systems still rely heavily on machine testing and artificial decision-making. As tasks get longer, skills combinations become richer, and the cost of machine data collection is high, and how to get limited human participation where it is most needed becomes the core of the RSI. To address this issue, RoboCoach trains the weak submission strategy in World Model Positioning tasks, guiding humans to teach and update strategy experts to improve their ability to maximize tasks with limited schedules.
On 5 October, Mulligan, released by the Stanford Shuran Song and Chelsea Finn teams, made the same judgment: uniform collection was a costly waste. The difference was that Mulligan focused his budget on the observed failure, while RoboCoach focused on the weak subtasks that positioned the world model.
The first author of this article is Zhang Wu, professor of automation at the Chinese People's University of China (CPC), project leader Xu Mandi, assistant professor at the Center for Cross-Information Studies at the Chinese University of China.
Let's see what we find.
The core of the RSI is to improve effective iteration efficiency under a fixed budget. Real robot verification and high human demonstration costs require limited resources for the most efficient use of improvements. RoboCoach increased the success rate of Franka from 13.3% to 75.0% under a limited artificial demonstration budget, and AgileX from 40.0% to 83.8%, with significantly improved efficiency compared to baseline methods.
2 The value of embedded data depends on its complementarity with the existing embedded base model. The value of embedded data is the ability to provide new capabilities for the model's current shortcuts.
3 World models learn the general laws of dynamics, whereas action strategies require both real-time and task specificity. World models use the removable dynamic laws behind different robots and tasks to replace real-time verification in the closed-loop era. But robotic strategies need to be kept real-time in real-time execution, while different tasks often respond to highly specific skills.
4 Combination of long-range tasks is a generalized problem that can be efficiently achieved through subtasking strategies. Long-range tasks have an exponential combined complexity, and the enhancement of targeted subtasks can greatly reduce the need for human demonstration data. RoboCoach achieved an average success rate of 35.0% on four already-skill rearranged, cross-tasking or directly interdependent held-out tasks, with the corresponding shared strategy baseline of 0% through the expertise of replicating subtasks.
The difficulty of having a physical RSI
How to complete long-range missions by correcting the mistakes of corporate strategy
Many robots have learned to grab, place, press, pull, and so on. The difficulty is that the real need is usually not to perform an isolated action, but to organize multiple actions according to the goal. For example, it is the same with the push of a button, sometimes in green and sometimes in red, in reverse order; it is also the same with the push of a button, which can be placed in a drawer, shoebox or cup, and also with the subsequent closing of the door.
In terms of the number of combinations, if there are m-class skills, and the length of the L-tasks can select any one of them at each step, then theoretically the sequence of skills that can be ranked is m^L. The real task also adds to changes in the object, position and execution state.
The starting point of RoboCoach is to bring the learning needs of the entire task back to the replicable subtasks. Long-range tasks can be long, but one improvement does not necessarily need to cover the entire route. If the robot has been steadily completed taking and placing and only repeatedly making mistakes on the torch, the next round of training should prioritize improving the pulling skills and allowing this ability to serve the other task combinations.
02 RoboCoach design ideas
Shared forecasts, expert execution
RoboCoach will predict and execute separately: the General World Model accumulates knowledge of inter-tasking interactions, modular strategies undertake execution and update in specific tasks.
General World Model: Understanding the Physical Changes in the Multiple Roles and Tasks of Robots
CoachWorld is a team-built motion-condition world model. It receives historical vision observations, mission texts, robotic movements, and camera geometry information to predict what subsequent vision changes these movements will lead to.
In order for the same model to cover single-armed and two-armed robots, CoachWorld used a unified two-slot end executor: each slot recorded three-dimensional position, 6D rotation, claw state and efficacy marking.
The geometry of the camera is responsible for matching the movements in the robot's coordinates to the images in each perspective: the same movement in the robot's coordinate system may move in completely different image directions from different points of view, and the distance may vary. First, project the three-dimensional end trajectory to the corresponding camera, and then use the trajectory and camera information to bind the video to generate the action instructions based on the corresponding image location.
This design allows for data from different entities to enter a shared predictive space. The CoachWorld is based on the Wan2.2 TI2V-5B initialization, using nearly 500 hours of interaction data training, which includes not only successful demonstrations, but also failures, free interactions and sub-optimal trajectories.
CoachWorld aligns the end-user movements with the camera geometry to predict the next video on the basis of historical images, text instructions and future motion trajectories. Single-arm and two-arm movements can be done using a single interface.
Expert on modular strategies: real-time and task-specific interactions with physical tasks
The world model must predict the consequences of the action, and the execution strategy must give the next action in the current entity, task phase, and control time sequence. The former needs to fully share knowledge of interaction, while the latter needs to retain locally adjustable parametric positions and context interfaces.
As a result, RoboCoach organized expert skills that could be individually adapted on top of the shared VLA base. In specific experiments, Libero and Franka used the π0.5 base, while RoboTwin and AgileX used the MolmoAct2 base. Experts carried the ability to learn with an independent LoRA adapter.
For example, using the red button and using the green button to point to different objects, you can call on the same pressure specialist. The current task is matched by a replicable module: the subtasks tell you what to do, which object to do, which expert is responsible for executing, and receive new demonstrations of the behavior.
Modularization is the adaptation of the strategy to the updating position, rather than splitting the same physical interaction capabilities by task name. One improvement in pressure capability can be continued in different buttons and task order.
During execution, the expert switches to the task; when learning, the new model enters the corresponding expert. Thus, the task breakdown at the same time determines how to complete the task and how to continue learning after failure.
RIDI: How does a robot get a specific supplementary lesson plan?
RoboCoach organized the training process into a four-step process called Route Yoga Imagine Yoga Diagnose Yoga Improve RIDI.
Route: Turn long distance instructions into executable submission routes
The router determines the next submission based on the task instructions, current visual status, already-existing skill vocabulary, and completed task prologue, and sends it to the corresponding expert.
Imagine: Let the current strategy run in a closed-loop world model
Current skills experts first observe the output movements, which are converted into the end-user trajectory, and CoachWorld predicts the next video after executing these movements; the generated image returns to the strategy and determines the next move.
Diagnose: The first link to stop with a progressometer
Each time a prediction video is generated, the progressometer combines the current submission and the visual sequence to give the completed progress. Once the threshold is reached, the system enters the next submission. If the deadline is not reached, the first submission and its experts in the execution are recorded.
Failure to take tea bags: The actual execution (up and down) and the CoachWorld imagined execution (below) did not take tea bags, the progressometer never confirmed the completion of the tea bags, and the system requested the sample.
Improve: Turn failure statistics into data requests that humans can execute
The system summarizes the first overtime records of different missions into a scorecard. It not only shows the overall imagined success rate, but also shows how the subtasks arrived, where they failed, the progress curve, and the representative failure video.
Scorecards can be drawn from imaginary success rates and high-frequency overtime targets for each task to a specific failure: viewing video at the time, progress curves, and expert switching records. Each data collection request corresponds to execution evidence.
After orientation, only updates are made to pick up the experts, placing and pouring experts; the actual execution with CoachWorld is done by taking tea bags, putting in cups and pouring water in succession.
By this point, the core logic of RIDI is complete: translate the failure of a complete task into the question of which subtasks require additional demonstration, from which experts to learn from. Next, it is necessary to answer through experiments whether the trainer sees fit, teaches correctly, and ultimately can make the robot really strong.
03 RoboCoach
Modular verification
The team then experimentally verified the following modules: whether the world model followed the motion and predicted physical interactions and reflected actual deployments, whether the progressometer could judge the completion of subtasks in these images.
1. World Model: CoachWorld predicts the execution of longer moves
The team made a complete segment prediction on a held-out test set. All models used consistent historical images and future action conditions, compared visual accuracy, tracked end-user trajectories, and the degree to which the physical laws and instructions of artificial assessment met.
In the DROID-180 assessment set, CoachWorld, which trained in a mixed field, excelled in six of the seven main indicators, including LPIPS 0.0996, FVD 51.71, and three end-to-end trajectory indicators and instruction-following scores. In Lab Franka-180, the LPIPS is 0.1712, which is also the lowest value in the comparison method.
The project's homepage also provides examples of action controls, as well as a long-term contrast between real execution and model prediction. The longest closed-loop example shown is 85.8 seconds: the model segment generates a follow-up image and continuously feedback the prediction results to the strategy.
Connect the computer and pull the rope: real execution (left) with the CoachWorld closed-loop prediction (right), full 85.8 seconds.
At the same time, the team further measured completeness of the mission as a measure of success, comparing the results of the strategy in the closed-loop test run in CoachWorld and deployed in the real world in a combination of 22 mission-driven strategies from LIBERO, RoboTwin, Franka, and AgileX.
Finally, the Spearman correlation coefficient of 0.840 and the Pearson correlation coefficient of 0.820 were obtained. This suggests that the success rates of the world models in the task portfolio assessed better reflect the performance differences between the strategies. RoboCoach can therefore first visualize tasks and behaviors that require prioritization by performing screening before a large number of real machines can be replenished.
The success rate of the imagination and the success rate of the actual deployment in a combination of 22 mission-driven strategies. The four types of environments use the same coordinate system and adaptive trends to demonstrate the ability of world models to depict differences in the performance of strategies.
2. Progress meters: provide signals for completion of submission-level tasks
RIDI is still in the process of deciding when to switch experts. If the tea bag is not yet fixed, the subsequent moves will start from the wrong state; the clutch has been completed but the delay in switching will also hinder the progress of the task. Therefore, we will evaluate whether the current subtask can be completed in a timely manner.
Under the same evaluation protocol, the team compared six methods of determination. All methods only read the current moment and previous observations, finishing the thresholds after being calibrated on independent data. The F1 evaluation in the table below evaluates whether the current subtasks have been completed between adjacent phases, and whether a binary decision should be made to switch the thresholds.
LIV、Contrastive λ、TOPReward and SuccessVQA are adapted according to the paper's Unified Cause Observation protocol.
Among them, RoboMeter is a method of making real-time progress judgments based on visual trajectories and is used in RoboCoach's progress meters.
04 Three rounds of correction, limited demonstration
How much is the overall mission success rate going to increase?
Four assessment areas, a unified new demonstration budget
The simulation consists of 10 long-range tasks in LIBERO-Long and five two-arms tasks in RoboTwin 2.0. Two sets of simulation environments involve object placement, switch drawing, operating boilers, placement of blocks, rolling bowls, etc. The real robot consists of three tasks in Franka Research 3 and four tasks in the AgileX arm platform.
Each method is based on the same initial strategy, with three rounds of updates under a fixed budget. Each round simulates a new 100 qualified submission demonstrations, each real-machine platform adds 50 items. The experiment compares how the same amount of new data is allocated and used by who, rather than simply increasing the amount of training.
1. Complete results from four assessment domains
On Franka, RoboCoach's three rounds of results were 13.3% → 46.7% → 68.3% → 75.0%; under the same data budget, the average collection baseline was 13.3% → 15.0% → 20.0% → 30.0%.
On AgileX, the change in RoboCoach was 40.0% → 72.5% → 83.8% → 83.8%, while the baseline was 40.0% → 48.8% → 46.3% → 47.5%.
LIBERO, RoboTwin, Franka, AgileX, and the Complete Task Success Rate Curve across the four rating domains. Under the same three rounds of budgeting, RoboCoach's overall performance was most prominent.
2. What specifically replaces different missions strategies
Just looking at the average value of the platform will still obscure changes in the level of the task.
For example, the initial overall success rate for tea-making tasks was only 10%, and after three rounds, RoboCoach reached 90%, while the base line was collected at a flat rate of 20%.
05 Skills after reinforcement
Better combined generalization capabilities
If each skill could only be used on a fixed route, meeting new instructions would still require re-collecting long-range samples. The real attraction of modular chemistry learning is that local reinforcements can be replicated by new task combinations.
The team constructed four complete mission routes on two types of real robots that did not participate in strategic training and mentoring, exploring long-range sequences, cross-task spellings, and skill rearrangements:
The average success rate for the four new routes was 35.0%; the baseline for identically collecting and updating shared adapters was 0% of the four routes. In these routes, RoboCoach succeeded 13 times in 20 virtual machine tests.
Four previously unseen mission routes. Experts who have been guided by orientation have been rearranged into longer operational chains, different task spellings and new sequences.
This set of experiments confirmed another point that the team insisted on in the design: skills should be the unit of ability to be repeatedly called upon, to be able to continue to generate value in new tasks. For long-term tasks, improving a weak skill will benefit not only a training route, but possibly a combination of tasks that will require this behavior in the future.
06 Prospects
From active substitution correction to physical RSI
In an effort to answer the question of how to efficiently and proactively correct bug fixes, Mulligan proposed by Shuran Song and Chelsea Finn's team to turn "where the failure starts" into an optimization variable, so that each round of collection falls where the strategy really rolls. When the failure is concentrated in a few initial states, from these states, the ability to change budgets is scattered faster than average. RoboCoach turns "which sub-skills fail" into a query object, allowing each lesson to be entered into its own expertise.
We look forward to the next step in the RSI system, a more inhumane closure: a world model with strong physical reasoning capabilities that decides "where to adopt" with the expectation of failure, and decides "what to adopt, who to update" with progressive monitoring and diagnostics, while humans only need to demonstrate one of the most critical steps.
Paper and project resources
RoboCoach: World Models as Active Coaches for Compositional Robot Skills. arXiv:2609.39685
The paper is written in English:
The project's homepage and more videos:https://robocoach-ai.github.io/
CoachWorld open source code:https://github.com/RoboCoach-AI/CoachWorld
Pre-training weight:
Related works mentioned in this article are: Lars Ankile, Perry Dong, Rohan Bhowmik, Aneesh Muppidi, David D. Yuan, Shuran Song, Chelsea Finn. Mulligan: Performance-Guided Data Collection for Efficient On-Robot Learning. arXiv:2610.05882(2026)
Mulligan's paper in English:
© THE