Recently, a very science-fiction video has been circulated on the Internet:
The film begins with a natural language instruction: use on-site materials, model and verify the six-level pH of the rainbow. No one is able to remotely control the materials on the experimental deck, the system first identifies the materials on the deck, and then lets the quadruped robot, humanoid robot and two-armed robot scan, transport, connect to the deck operation in sequence. Three robots of different configurations share context on the same task.
The true performance of the system was after a deviation occurred in the middle of the mission. The color of the sixth test tube did not reach the target, the system did not complete the pump action when the task was completed, nor did it come back from scratch, but instead maintained the on-site state, supplemented the buffer liquid and re-tested.
The PhyAgentOS v1.0.0 provides a traceable engineering foundation: an open-source physical intelligence body Harness developed jointly by HCP Laboratory, Shenzhen National Laboratory, and X-Era Lab. It is compatible with VLA, world models, simulations and virtual machines, and provides a unified execution base for heterogeneous robots, also known as RSI (Recursive Self-Improvement, recursive self-improvement). The project opened in July 2026 and released its first stable version at the end of August; as of now, GitHub Star has reached its 2100 threshold.
GitHub：
Website of the project:
Technical report:
01 Three robots turned on power and didn't come back after a failure
PhyAgentOS REPLAN3 is a live demonstration
In this four-minute presentation, the user gives only the final goal and does not write an action script for each robot individually. After the system completes environmental perception and material positioning, the scanning, transportation, connection, and distribution steps are assigned to quadruped robots, humanoid robots, and arm robots.
After the first five trials were successfully completed, the colour of the sixth trial failed to match the target. Here, if only the control signal is seen, the robot has completed the operation; but Verifier integrates the field image with the test results, and the judging task is still incomplete. The system adds PlanRevision to the same AgentTask, retains the previous action and current environmental status, replenishes the buffer and the tester and checks again. Until the pH test reaches 11.03, the task is marked as passing.
This also concretizes the problem that PhyAgentOS has to solve: physical intelligence cannot only send out the action, but also to know if there is evidence at every step, whether the end result is met, and where to go after failure.
Figure 1 PhyAgentOS v1.0.0 Overarching architecture: Forge Unified Physical Execution, Verification and Experience Backflow consists of two credible locks.
02 Results Verification and Control Restoration: From the end of the hunting operation to the completion of the hunting task
Traditional control links often view the finished signal that the device returns as the end of the task, but the claw that is closed does not prove that the target object has been captured successfully, the claw instruction has received the claw, nor does it prove that the user's goal has been reached. PhyAgentOS disconnects the execution facts, field evidence, and task conclusions: Query, Action, and Session record tool calls and execution states, the Evidence module saves images before and after the action, such as the results of the detection and the state of the device, the task level Verifier re-dependent on the goal, the success criteria and the timing of the evidence, and gives the success, failure, or replan required conclusions.
Figure 2 Exercise fact-based and on-the-spot evidence to enter Verifier and output verifiable task-level conclusions.
When the verification is not passed, the recovery process still belongs to the original AgentTask. The system adds PlanRevision based on existing tool trajectories, evidence chains and the latest environmental status, rather than re-creating tasks or blind retrying. This both maintains state continuity and limits the recovery range and number of retrying attempts.
The benchmark test further quantified the mechanism. In the 2000 episode of LIBERO, the first success rate of X-VLA was 97.3%; PhyAgentOS analyzed and recovered 54 first failed samples, saving 26 and ultimately raising the success rate to 98.6%. The system did not modify the strategy model code, nor did it record the recovery process as a new episode. In the five-step long-range mission of CALVIN, the complete chain improved to a maximum of 4.1 percentage points; in RoboCasa365, the RLDX-1 and WorldDreamer improved to 7.2 and 8.4 percentage points respectively.
Figure 3 First attempt and Final outcome are separated statistics to distinguish between the original capabilities of the strategy and system recovery gains.
03 Experience Backflow: Skill Continued Development Driven by Validation
The result verification not only determines whether the task is completed, but also the first gateway to experience entering the system. PhyAgentOS uses the complete task verified as a basic sample to organize targets, plan revisions, tool trajectories, evidence fingerprints and final conclusions into a TaskEpisode and then enter the experience refinement process. No evidence supports or execution records that have not yet been closed will not be directly converted into new capabilities.
After experience refinement, two types of candidate outcomes are formed: stable, reproducible workflow can be promoted to SkillCandidate, recurring problems that can be attributed to workflow can be grouped into Lesson. Environmental abnormalities such as device downtime, network vibration, sensor noise or insufficient evidence retain only diagnostic information, and do not enter the laboratory library. Skill Runtime is responsible for checking skill products, depending on the environment, health status and life cycle, ensuring that pumps know how to do pumps and pumps are currently unable to do pumps.
The candidate experience has to go through multi-task verification, structural validation and security checks before it becomes effective. The system filters specific coordinates, credentials, temporary device IDs, and any instructions attempting to bypass Forge or Verifier to avoid a chance of successful contamination.
Figure 4 Verified AgentTask is entered into Skill or Lesson only through TaskEpisode, reflective attribution and multiple independent support.
04 Forge execution framework: physical movement input of unified heterogeneous robots
Forge is a general-purpose robotic control framework for Physical AI that uses unified robots, sensors, and environmental abstraction to organize hardware and algorithm strategies into compostable, replicable, and structured calls.
In real-world scenarios, robots rarely come from the same manufacturer. In warehouses, quadrupeds may be responsible for patrolling narrow corridors; mechanical arms are responsible for picking up and placing in fixed locations; humanoid robots are better suited to operating cabinets, doors, or tools that were originally designed for humans. They use different control interfaces, sensors, and software environments, and in the past, they often had to be developed and modulated separately for each robot to work together.
A central highlight of PhyAgentOS is the unified protocol for connecting different robots to the same task system. The upper-level Agent does not need to understand the control details of different robots one by one, only to call a standardized Tool; the lower-level Forge is responsible for identifying device capabilities, routing tasks, issuing commands and managing execution status.
For example, the shop system gives the robot the task of completing the A-Zone shelf inspection and inventory checkout, four-legged or wheeled robots can be responsible for moving and collecting images, the visual ability is responsible for identifying the shelf status, and the mechanical arm is responsible for removing or replenishing in conditioned scenarios. The system focuses on whether the task is completed, rather than the operator having to manually switch off three sets of device controls.
The value of a unified protocol is not to make all robots the same, but to allow them to play their respective roles in a traceable chain of tasks. When new robots enter, they can be called upon by upper-level tasks as long as they declare their capabilities in a unified manner; when replacing a robot's body, model or simulator, there is no need to synchronize the entire task logic.
Here, PhyAgentOS unravels model decision making and action execution: the upper-level Agent is responsible for task understanding, planning and strategy adjustment, Forge as the only physical action execution input, responsible for detection, routing and execution state management capabilities. Gateway unifies registration and management skills, ToolEndpoint maps tool calls to specific robot actions, Dora data streams carry real-time data and control links.
Markdown files such as AGENTS.md, SKILL.md, etc. continue to carry readable knowledge of the person and Agent; real action is unified through the Forge Tool API; AgentTask, PlanRevision, Evidence and verification results write structured fact records. Knowledge, execution, and fact fall into appropriate carriers, and the system no longer relies on routing files to drive physical movements. Even if it encounters overtime, cancellation, or unknown results, it verifies the on-site status and decides on subsequent actions, reducing the risk of repeat execution.
After the upgrades to the Figure 5 protocol, knowledge, execution and fact layers are stored, and Markdown no longer assumes the roles of scheduling, status and logs simultaneously.
05 Using Tool and Skill to combine atomic capabilities to solve complex long-range tasks
Work in the real world is usually not a move. Inspection shelves need to be moved, identified, photographed, judged, reviewed; supplies need to be confirmed, picked up, avoided, placed, checked out; a change of model on the production line may involve moving materials, opening workpieces, charging, testing and abnormal handling.
PhyAgentOS breaks down these actions into a call-able, verifiable tool. For example, a readable status bar moving to a shelf bar to capture images of a material bar to grab a material bar and place a product bar to verify results can be a tool. After multiple validations, a process that can be reused can be further submerged into a Skill.
This free combination of Tool and Skill means that the system doesn't have to rewrite a rigid script for each complex task. Faced with tasks such as scrolling the shelf in zone B and dealing with shortage of goods, it can combine the ability to move, scan, identify, grab, and replicate depending on the situation in the field; it can also adjust the problem link without having to start the whole process from scratch if there is a blockage, product error, or grab failure at some stage.
This is also the premise that complex long-range tasks can go to practical use. Robots do not need to remember all processes at once, but are able to call on modules of capabilities that have already been verified, subject to task objectives, and continuously advance according to field feedback.
06 Ecology and Access: Keeping Models, Skills and Robots Combined
The Unified Tool Contract reduces the cost of cross-configuration adaptation. According to the project team, access to a new robot configuration usually takes only 5 to 10 minutes; when replacing a model, simulator or robot entity, the upper layer Agent does not need to be synchronized with the task link. The officially supported robot configuration has been expanded from 19 to 43 models, covering machine arms, quadrupeds, humanoid robots, mobile operating platforms and agile hands, 9 of which support real machine operation. The official version provides 9 nodes, covering VLA, world models, visual perception and simulation learning, and 11 other nodes are included in the development plan.
PhyAgentOS is open source under the MIT protocol, and the core warehouse has been moved to the PhyAgentOS organization and renamed PhyAgentOS-core. Once the model Provider is configured, even without real hardware, the General Agent can be run first.
git clonecd PhyAgentOS-corepython -m pip install -e .paos onboardpaos agent
The core change in PhyAgentOS v1.0.0 is not to add a layer of model packaging, but to complement executable, verifiable, recoverable, scalable Harness, Agent, Forge, Evidence, Verifier, and Evolution, respectively, with responsibility for planning, execution, retention, acceptance, and experience. This set of boundaries determines whether a Demo can be transformed into a long-term, stable system capability.
Industrialization will not be completed suddenly because of the release of an open-source system. The reliability of the robot's body, on-site security, deployment costs, data quality, network conditions, and connectivity to existing business systems still determine whether or not artificial intelligence actually enters the production process.
But the direction of the industry has become increasingly clear: the competition is shifting from whether robots can complete a stunning demonstration, to whether robots can complete a stunning demonstration, verify results, handle exceptions, and continue to reduce the risk of artificial intervention.
In retail stores, a shelf inspection after closing is a typical scenario. The robot moves along the corridor, reads the status of the shelves, discovers a missing or misplaced item at a certain location, and then calls the corresponding equipment to complete the review or replenishment. It does not initially have to take on all the work of the store, but can first complete the stable closing cycle within a region, a category of goods, a clear time window.
In factories, the production line switching model is also suitable for this type of collaboration. Mobile robots send the material to the work site, the mechanical arm continuously completes the load or assembly, and the visual system is responsible for checking the results. Once a part is found to be in the wrong direction, displaced or assembly incomplete, the system maintains the on-site state, returning only to the wrong link, instead of having the entire production line stop and restart.
In fact, as the main R&D team at PhyAgentOS, X-Era Lab is bringing the system further into real-world industries such as smart retail, industrial manufacturing and residential services, enabling Physical Agent to complete verification, feedback and capability iterations in continuous operation, gradually evolving from single-point Demo to long-term deployable, evolving and scalable system capabilities.