The Embodied RSI Debate: Are Robots Self-Evolving or Just Repeating Practice?

In video games, players have an almost unbeatable privilege of having unlimited read files (env. reset).

In robotic simulation environments, some agents can also find better operating strategies by exploring tasks in advance, resetting environments repeatedly, and modifying scripts.

But what if we don't give robots a chance to answer the questions in advance?

Recently, Auriga, the general-purpose Harness framework launched by Intelligent Design, topped the RoboDojo list under the Zero-shot evaluation protocol, which does not target target task pre-optimization, does not rely on expert demonstration, does not allow single execution and does not allow environmental repositioning. At the same time, Auriga also led the way in the RoboCasa 365 Target 50 evaluation with the same set of Harness frameworks.

Auriga has been able to overcome his opponents without having to brush them down. How did Auriga do that?

The answer may not lie in the standard answer to remembering more tasks, but in building a set of physical execution systems that can be replicated across tasks. This also raises a deeper question: are robots optimizing the answers to a question or evolving the ability to solve a problem?

1. The three levels of the RSI are:

The answer to evolution, the process of evolution, the system of evolution

The evolution of a robot's capability depends not on how many trials and errors it has made, but on what changes with each improvement: the solution to the current task, the one-time operation strategy, or the ability of the system to be replicated by different tasks in the future.

Based on this perspective, STAR Intelligence divides the path of development of the Physical RSI into three levels:

• L1: Task-level search, that is, evolutionary responses. The system searches for better optimization for a specific task by repeatedly resetting the environment, modifying scripts or adjusting control parameters. It is able to continuously improve the performance of the current task, but often needs to be re-optimized for new tasks.

• L2: Single-line adaptation, an evolutionary process. The system no longer relies on repeated repositioning, but on continuous execution, based on observational and feedback adjustment movements. Improvements occur during the execution of current tasks, but these adjustments do not necessarily overwhelm the general capabilities available for other tasks.

• L3: System-level capability improvements, i.e. evolutionary systems. The system identifies recurring commonalities issues from the interaction of different tasks, and further improves the underlying general tools, control of native languages or skill mechanisms. Unlike L2, these improvements are retained in the system for subsequent assignment calls, and verify their migration value through new assignments.

The change from L1 to L3 is not just a trial-and-error approach, but also a range of ability improvements: from an answer to a question, to a process to be executed at a time, to a set of systems that can handle different problems.

Auriga opts for the system-level generic capabilities represented by L3: not to reshape the strategy for each new task, but to allow the same set of Harness to support the perception, execution and correction of errors online in different physical tasks.

The real evolution is not that robots remember more and more answers, but that they have a set of solutions that don't have to be reinvented for every new problem.

2. Auriga：

General Harness for building large language models into the physical world

Auriga is a general-purpose Harness framework developed by Star Spy that integrates physical perception, motion expression, execution verification and mission status management into a unified interactive system that supports large language models for autonomous planning, execution and online recovery in unfamiliar tasks.

In this architecture, model-dominated task decision-making, Harness provides quantifiable physical states, complete motion expressions and verifiable execution processes, enabling high-level reasoning to be transformed into continuous robotic operations.

The system architecture revolves around three core capabilities:

• Perception as Measurement: A combination of three-dimensional reconstruction, visual division and robotic mechanics, converts images into spatial information with real physical measurements that can be queried and verified.

• What You See Is What Executes: Through a unified execution gateway, movements generated by the model are converted into visualized trajectory previews and continuous verification during actual execution, abnormal interruptions and online recovery.

• Failure as Evidence: structured recording of model intentions, planned trajectories, actual movements and execution results, supporting long-range mission status tracking and historical retrieval, and improving the accumulation of physical evidence for cross-mission capabilities.

Figure 1: Auriga General Body Harness overall architecture and execution processes. Through a unified perception, planning and execution interface, high-level decisions are transformed into verifiable physical actions. The system maintains task status during duration sessions, locates targets through three-dimensional geometry tools, and performs trajectory previews before execution.

3. From Understanding to Doing:

How does Auriga open a complete closed loop of physical execution?

Getting the model to understand the task is just the first step. The real challenge is how to translate the model's reasoning into accurate, reliable physical actions and correct errors in a timely manner when they deviate from what was expected.

Around this problem, Auriga integrates three-dimensional perception, action planning, execution verification, and historical feedback into a general loop, allowing the model to not only determine what a mouse can do, but also to check if a mouse can do a mouse, judge if a mouse can do a mouse, and decide what to do with a mouse after a failure.

Perception is measurement: an eye with a pair of scales for big language models

Large language models can understand objects and their spatial relationships from images, but robot operations require precise three-dimensional position, size, and gesture information.

Auriga integrates VGGT-Omega, SAM 3.1, camera identification and multi-point geometry, converting visual observations into spatial information with real physical dimensions. The model can actively query the location, direction and size of the target, and use triangle measurement and three-dimensional heavy projection to check critical contact points.

For captured objects, the system can also combine wrist cameras and claws to measure changes in the position and posture of the object in the hand, helping the model determine whether there is slippage.

Figure 2: Physical perception of Auriga. Through three-dimensional geometrical measurements, tracking and historical failure records on hand objects provide a questionable physical state for big language models.

What we've seen is that every move is previewed and re-verified

After accurate spatial measurements, the model also needs to specify how the robot touches objects, how it moves, and how the two arms collaborate.

Auriga supports a model that directly specifies the finger's contact position, precision of movement, speed of movement and simultaneous coordination of the two arms. The model can clutch, lift, move and place the tissue in a complete multi-stage sequence of movements, and set speed and precision requirements according to different operating stages.

Figure 3: Expression of Auriga's precise movements. By positioning the point of contact of the fingers, the precision control of the movements is synchronized with the timing of the two arms, and the intent of the operation is transformed into a multi-phase sequence of movements.

All physical instructions, whether from a direct model call or from a self-written program, are executed through a unified gateway, following the Preview-Execute-Verify process:

• Preview: Harness will compile the sequence of moves into visual trajectories, showing the path of the two arms movement, key gestures, expected time-consuming and potential collision risk. The model can check and adjust the plan before actual execution.

• Execute & Verify: The system performs the movements according to the preview trajectory and controls the state of the robot test. When the deviation exceeds the set capacity limit, it automatically interrupts the follow-up movements, returning the deviation position, measuring the results with the latest observations.

• Online Recovery: The model is re-planned based on physical feedback, continuing to operate from its current state, maintaining the progress of the completed task, without having to reset the environment.

Figure 4: Auriga's trajectory preview and execution verification mechanism. The system visualizes trajectory planning prior to execution, disrupts actual movement to expected state in execution, automatically stops when unexpectedly obstructed, and supports model replanning and continuing execution based on the latest observations.

Failure is proof: Let every physical interaction be traceable

In complex long-range tasks, the model needs to continuously track the progress of the task and update judgments on the environment and the results of the operation based on physical feedback.

Auriga establishes an explicit mission state and retrievable execution history. The system retains current plans, key focuses, and mission progress, while recording the model's action intent, planned trajectory, actual movement, visual observation, and execution results.

Even if the context is compressed on the model in the long-range task, the critical task state can still be maintained. When an abnormality occurs in execution, the model can trace back to previous moves and failures, analyzing the bias between expected and actual execution, and avoiding repeated ineffective operations.

These structured records also provide the basis for cross-task analysis, enabling the system to identify physical execution problems that occur repeatedly in different tasks, and accumulate evidence for subsequent improvements in general tools and skill mechanisms.

Auriga challenges RoboDojo and the RoboCasa 365.

How much functionality can a universal Harness enable a large language model to play in non-biological tasks?

Auriga was systematically evaluated on the basis of the GPT-6 Astra model, based on the RoboDojo and RoboCasa365 robot simulation benchmarks, focusing on its Zero-shot operation, cross-scenes migration and online recovery capabilities.

42 missions, 6300 tests: Auriga leads the RoboDojo list

The RoboDojo list covers generalization, precision operations, long-range tasks, memory and open tasks, with a total of 42 robot operations.

Unlike methods that rely on prior exploration and iterative optimization of targeted tasks, Auriga uses strict zero-shot evaluation guidelines:

• Zero-target task pre-optimization: No pre-developed dedicated scripts for test tasks and their layout, and all tasks share the same fixed Harness set.

• Zero-expert demonstration: No successful trajectory, video demonstration or task-specific code templates are provided, based solely on task descriptions and allowed real-time observational autonomous decision-making.

• Single-stage execution, zero-environment reset: Each case has only one formal execution. Even if there is a clutch failure or movement obstruction in the middle, it must continue to adjust from the current physical state.

In a total of 6300 complete test episodes of 3 random seeds, Auriga achieved an average mission success rate of 57.74% (SR) and an average process score of 64.00 (Score), both of which led the RoboDojo list.

From the five capability dimensions, Auriga excelled in generalization, memory and open tasks, with a memory task success rate of 92.99%. Compared to partially reliant on target task pre-exploration, Auriga still demonstrates strong integrated operations without prior optimization for target tasks.

Table 1: RoboDojo full-size rating ranking,† refers to using the GPT-6 Astra as the base model

Can a different scene, a different robot, and the same Harness still work?

Achieving a high success rate in a fixed scenario is only the first step. True generic ability also requires testing for changes in object layout, task structure, and even the robot's own body.

The first major test: randomization of object layout.

In RoboDojo's 12 generalization class missions, the research team further tested the post-randomization performance of the object layout. On a comprehensive consideration of the criteria and the G indicator of random scene performance and its differences, Auriga scored 46.3 points, leading the second place by 2.4 points.

The second test: replace a robot, replace a kitchen.

The team further migrated Auriga to the home mobile operating benchmark RoboCasa 365. The robot transformed from a two-armed desktop operating platform to a mobile robot with a single-armed Franka, and the mission environment also moved from the desktop to 10 unseen kitchens, requiring the completion of tasks such as switching cabinet doors, operating drawers, rotating rotors and home control, with some long-range combination tasks containing up to 15 subtasks.

In all 50 missions and 250 test episodes of RoboCasa 365 Target 50, Auriga achieved an overall success rate of 73.2%. Among them, the single-skill success rate reached 92.2%, and long-range combined missions reached 62.5%.

Figure 5: RoboCasa 365 Target 50 success rate comparison

From two-armed desktop operations to mobile single-arms operations, from fixed desk to unfamiliar kitchens, Auriga demonstrates the potential for general physical interaction mechanisms to be replicated across robotic forms and mission environments.

A mirror: How can a robot save itself after a failure?

In real physical operations, objects may slip out of the claw, the mechanical arm may encounter obstacles, and the target may deviate from its original position due to contact. The key is whether the robot can detect actual results that are not in line with expectations and continue to solve problems from the current state.

Auriga's full execution record shows many different levels of online recovery behavior.

Recovery from Failure: Self-adjusted crawling and operation

When an object slips during grab or transport, the GPT-6 is able to use the latest observations and physical feedback from Harness to reposition targets, adjust grab positions and movements.

In toy crawling, conveyor belt sorting, and bowl stacking tasks, models continue to operate from their current physical state by changing the crawling area, adjusting the mode of transportation, and rearranging the stacking order, respectively.

From local error-correcting to strategic replanning: automatically changing operational schemes

More complex physical accidents often cannot be solved by simply retrieving or minimizing coordinates. When environmental conditions or operating conditions change, the GPT-6 is able to reorganize subsequent missions based on physical evidence returned by Harness.

In the object container mission, the model actively adjusted the mechanical arm partitioning and placement path after displacement due to a collision; in the conveyor belt mission, the model converted the suspended load to a first stabilizing basket casing, and then placed the object; in the charger insertion mission, the model repositioned according to the mm-level position deviation, completing the precise insertion.

From modifying local actions to adjusting overall operating strategies, Auriga enables the model to continue to advance tasks in the changing conditions of the physical world.

From a one-time failure to a complete Harness upgrade

If online recovery solves "how the current task failed", then another question worth considering is whether the physical difficulties that repeatedly arise in different tasks can further drive the improvement of the entire execution system.

Auriga's research and development process provides a concrete example.

In the 13 generic Harness capability iterations promoted by the Star-Spangled Bones research and development team, Auriga gradually transforms the common problems exposed to different missions into replicable perception, control and execution verification mechanisms, continuous improvement of movement previews, arm coordination, three-dimensional geometry measurement, state tracking, and obstacle inspection.

As these capabilities were gradually added, the average score on 42 RoboDojo missions increased from 45.5 on the 4th edition to 64.0 on the final version, and the success rate increased from 36.2% to 57.7%.

Figure 6: The 13 Harness Evolution by Auriga

These improvements eventually settled in the shared Physical Harness. The system was frozen before being formally assessed and its replicability was tested through various RoboDojo missions, randomized layouts, and another type of robotic environment for RoboCasa 365.

This process demonstrates an important technological path: transforming recurring physical execution problems in different tasks into underlying capabilities that can be shared for more tasks.

From Zero-shot execution to General Physical RSI:

Building the next level of infrastructure for physical intelligence

Auriga's performance on the RoboDojo and RoboCasa365 initially confirmed the possibility of a general large model completing non-biological tasks by sharing Harness. But precise assembly, complex contact with long-range operations remain important challenges. Further enhancing physical performance and allowing robots to continuously accumulate experience from interaction are key directions for the next stage of stellar intelligence.

From the General Large Model to System 1/System 2: Let's make reasoning and execution synergies

Future robots will be able to achieve layered intelligence through the collaboration of System 1 / System 2: System 2, represented by generic models such as GPT-6, is responsible for task understanding, global planning and exceptional decision-making; System 1 is assumed by action models such as VLA, WAM, and is responsible for rapid, continuous movement and contact control.

Star Interactive intends to use Auriga as a common physical execution infrastructure that connects the two, enabling high-level decision-making and low-level controls to form a reliable loop through unified physical interfaces, trajectory previews, and verification and feedback mechanisms.

Let the brain be responsible for understanding and making decisions, let the brain be responsible for quick execution, let Harness be responsible for connecting, verifying and feedback.

From Failure as Evidence to Failure as Improvement: Moving to the Common Core RSI

If System 1 / System 2 synchronization solves the question of how to do a better job, then the general Physical RSI answer is another longer-term question:

Can robots continue to improve their ability to solve new problems by continuously performing tasks?

Currently, Auriga has been able to convert model intentions, execution trajectories and physical feedback into traceable evidence that supports autonomous error correction in single tasks and provides the basis for cross-task analysis.

The next step is to drive the evolution from Failure as Evidence to Failure as Improvement: allowing models to identify commonalities from the failures of different tasks, independently build, modify and validate the general Skill, continually improve shared physical execution systems, and improve migration values through unknown task checks.

From the online correction of single execution errors, to the accumulation of multi-tasking capabilities, and to the autonomous improvement of the execution system, this is the long-term direction of Star Spy's exploration of the general Physical RSI.

What is the meaning of this verse?

The physical world does not have infinite readings, but intelligence should not start from scratch every time.

The real evolution is not to repeat the same problem a hundred times, but to allow robots to grow from the previous one hundred problems and have a stronger problem-solving capability when faced with the first one hundred and one new problems.

• Project Page：

• Technical Report：

• Code：

• Demo：

© THE