Recently, a simulation assessment report published anonymously on GitHub revealed that a deep-water bomb was dropped on a smart circle: the architecture of the hybrid GPT-6 Astra was 62.6 points, and the second was only 38.26 points.
The 64% performance gap, which indicates that the gap is no longer modestly optimised, can be said to be a reduction in architectural impact.
The study did not use the GPT-6 Astra to directly manipulate the mechanical arm, but it did a smarter thing: using the GPT-6 Astra to provide semantic-level judgments and corrections, interacting with physical space at π0.5 and acting as a precursor to complementary structures.
The in-depth evaluation of the Astra model by the Galaxy General team revealed a major breakthrough in System 2 universality. Although the trajectory quality and real-time performance are not high, zero-sample robots have a high success rate in completing all new tasks and have a strong error recovery capability.
It seems that physical intelligence is no longer far away from us, and the universality of robots has jumped to a level similar to ChatGPT with trillions of model parameters.
How is this evaluation done?
The authors of the report came from a team of researchers at Galaxy General, the first of whom was a Ph.D. student of Galaxy General founder and CTO Professor Wang Chen. The entire assessment was a rigorously coordinated closed-loop controlling contrast experiment.
The study analyzed two closed-loop control architectures used to simulate two-arms operation:
Direct mode: Do not runπ0.5The GPT-6 Astra directly looks at the three-way camera image and the state of the robot, with the output of the two-armed endpoints combined with the claw, performing 1 to 5 control steps each time.
The mixed model: every moment of decision-making,π0.5 Create a 50-step candidate action. GPT-6 Astra looks at the same image, executes history, and the two-arm trajectory of the candidate action, and then selects one: either use it alongside.π0.5 The first 1 to 15 steps, or your own 1 to 5 step end position correction. After this section, look at the new image and decide again.
The architecture is actually very simple. One of the reports also mentioned that the TopK thought process of selecting a candidate has been discussed many times in industry discussions. But it really ran out of quantitative differences for the first time.
The results showed that the mixed model had a success rate of 48% and an average score of 62.60, which was significantly higher than the second place score of 64%.
01
RoboDojo: Significantly improved success rate after GPT-6 Astra mix
The first is the RoboDojo experiment.
The team selected 10 complex robot-operating tasks, including sorting tables, sorting objects by language, sorting numbers, containers, towers, maxi plates, folding clothes and bottles into barrels, and each task carried out five tests for a total of 50 times.
The experiment set two scenarios.
One of them is:GPT-6 Astra DirectAstra generates robot movements directly based on vision, body status, and mission objectives; the other isπ0.5+AstraAstra observes environmental and candidate movements and makes adjustments when necessary.
The difference is obvious.
Astra Direct has successfully completed 13 of 50 missions, with a success rate of 13.26%On average, 48 missions with a full Native rating scored 48 points.37.81。
After accessing π0.5, the success rate increased to 24 times.48%The average score has risen to.62.60More significantly, Astra has only modified all of the execution control steps.14.4%。
In other words, most of the action is still done by π0.5, with Astra primarily responsible for understanding tasks at key nodes, judging current status, and deciding whether it needs to be corrected.
This group of experiments first demonstrates that the two types of models are clearly complementary.
Astra is best known for its semantic understanding, visual recognition, spatial relationship judgement, and mission planning. In tasks such as crawling object classification, Astra Direct even scores 100 points; in the face of catching failures, object omissions, or environmental changes, it is able to re-observe the scene and decide what to do next.
But when it comes to tasks that require precise touch, steady grabbing and continuous control, such as towers, masquerades, and more, the shortcuts of large models are quickly exposed.
On average, Astra Direct has only one of the few tower missions in the world.Twelve minutesIt's a great way to get to know each other.0 pointsThe model can know where the pieces should be placed, but it doesn't have to be able to control landing points, friction and collisions.
After adding π0.5, the tower was upgraded from 12 to 64 points, and the MMA card was upgraded from 0 to 40, with significant improvements in tasks such as putting clothes and bottles into barrels.
This is the most important layer of information in the report: Big Models have been able to enter the robotic strategy layer, but understanding the physical world and operating in the physical world are still two different capabilities.
02
RoboLab: Further verification of skills in semantically understood tasks
The RoboLab experiment further demonstrated Astra's advantages over another class of tasks, exploring general robotic strategies for understanding vision, object relations, mission instructions, and spatial structure. The team selected 10 classes of tasks, each with five assessments, for a total of 50 times, including placing squares in containers, looking for and moving pumpkins in trash, stacking trees in a specified order, adjusting the direction of the cups, and placing different objects in a specified location, among others.
The results were remarkable.
GPT-6 Astra Direct was successful 49 times in 50 experiments, with a 98% success rate; Astra+π0.5 was successful 46 times, with a 92% success rate. As a reference, π0.5 in the same set of statistics was successful 18 times, Cosmos3-Nano-Policy 18 times, and DreamZero 17 times.
It is particularly noteworthy that Astra has no additional training for these specific tasks. The model continues to determine how the next step should be operated based on current images, robot status and natural language task descriptions.
In terms of mission-by-mission results, Astra achieved five complete successes in multiple missions, including placing squares in boxes, handling dumplings in garbage, placing objects according to relationships, stacking them in a specified order, rearranging the direction of the cups, etc. The only one that did not succeed was placing a grapevine box in a container tank five times, completing four.
The core capabilities demonstrated by this group of experiments are:Zero sample generalizationThe model is capable of identifying objectives, understanding spatial relationships, transforming language requests into actions, and continuing to adjust strategies after environmental changes.
However, this figure needs to be combined with an understanding of experimental calibration. The report makes it clear that RoboLab experiments only selected 10 tasks; each method has five final evaluation slots per task, and not a complete RoboLab ranking reproduction; the historical baseline and Astra do not strictly guarantee exactly the same initial state and control configurations, and some Astra experiments include authorized retests.
As a result, 98% are better suited to observing Astra's capability limitations on these tasks, not simply as a strictly conditional ranking score.
03
The GPT-6 Astra Force understands missions and discovers where the clutch is wrong.
Going back to RoboDojo's mission-by-mission results, we can see further where it is strong.
Astra Direct scored 100 points, five of which were complete, with an average score of 60 points, 40 percent, and a maximum score of 57 points.
Score Heat ratings success rate Heat ratings
These tasks rely heavily on language understanding, visual recognition, relationship reasoning and goal planning, which is precisely the ability of large models to excel.
The experiments also showed a large number of behaviours that showed less of a traditional robotic strategy.
For example, a model will actively change approach once it detects an inappropriate grappling posture; when the object is in good shape, it will re-observe the environment, find that the task has not actually been completed, and continue to adjust; during the packaging process, it will be able to detect missing objects behind the box; and when the blockage was originally planned, it will be re-planned according to new visual feedback.
Chart: In an experiment, Astra continues to judge the status of the task based on new images after execution and adjust the movements when necessary. For example, in object categorization tasks, the model detects that a local correction is currently required; in long-range tasks such as containers, the model continues to check the remaining targets in the scene and then decides the next action.
There was also a remarkable recovery in the digital placement task: the model re-understood the control feedback of the mechanical arm after being blocked by movement, adjusted the movements and continued to execute. It demonstrated the ability of the GPT-6 Astra to adapt locally to the control constraints, rather than just passively repeating the failed commands.
Chart: Digital Placement: Understanding the wonderful recovery after controlling feedback
These phenomena show that the Astra does not mechanically execute a pre-defined trajectory, but instead forms a closed loop in which the Astra observes the decision of the Astra to execute a re-observation of the Astra.
Models even actively select strategies that are not necessarily common in training data. For example, in the bottle-to-bottle task, Astra Direct tried not to grab the bottle one by one, but to sweep the bottle to the trash can. This scheme ultimately failed, but showed that the model could find new solutions on its own according to the target.
04
The real bottlenecks are still in physical interaction.
However, the study also revealed Astra's weaknesses.
Astra Direct scored an average of only 12 points on a tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower tower.These tasks require precise contact, steady grip, collision control, and continuous judgement of an object's stress state.In comparison to semantic and visual tasks, the big model here is obviously powerful.
A typical question is:Understanding a goal is not the same as being able to steadily complete a move.
The model may know where the log should be placed, but cannot guarantee that the time tower will not fall; it may know that the object should enter the container, or it may fail due to a collision between the claw and the edge of the container. Current controls adopt a phase execution, and the model usually has to wait for a phase to be re-observed after the end of the action, so problems such as sliding and collisions during the action execution process are sometimes not detected until the next round of feedback.
This is where the value of π0.5 comes from.
With the addition of π0.5, the ratings were increased from 12 to 64, with a success rate of 60 percent; the ratings were increased from 0 to 40; the ratings were increased from 40 to 100; and the ratings were increased from 36 to 100.
π0.5 is a pioneer in the field of robotic motion data training, whereas Astra is responsible for identifying tasks, judging progress and correcting errors at a higher level.
The study also analyzed some of the interesting but not necessarily robust or physically impossible situations that occurred in the GPT-6 Astra mode.Lack of the mature movement and object interaction precedents provided by the Physical Policy, the model needs to solve the details of gripping, supporting, touching, releasing and arm coordination on its own, and some seemingly reasonable and interesting schemes are unstable in actual execution.
Zero-shot behaviour: an interesting scheme, not necessarily a rubbish move
The GPT-6 Astra (direct) will attempt to sweep the bottle into the barrel with a mechanical arm, grab the tower board with one hand, or complete an operation with one arm that was originally suited to the two-arms collaboration. These are different ways of achieving the same goal, rather than simply misunderstanding the task.
The question is:The ability to propose an operation that is likely to be effective and to perform the operation steadily is at two different levels.In the absence of the support of mature action pioneers, a program may neglect reliable grasp gestures, sufficient support or appropriate release times, making it difficult to convert locally feasible actions into stable tasks. The value of an embodied Policy lies not only in providing a response to a move, but also in providing execution patterns shaped by interactive experience to build a semantic-level program on a more reliable operational basis.
Chart: Zero sample scheme: try to sweep the bottle into the bucket with a mechanical arm
05
The cost of capacity building remains high
At the same time, the report emphasizes that the current programme remains significantly distant from real deployment.
The first is the cost of reasoning. In the RoboDojo experiment, π0.5+ Astra cumulatively consumes about.625 million tokensAstra Direct, on the other hand, is more than that.The number of tokens is 1.13 billion.Astra needs to repeatedly read the visual and historical context, and to reason longer, which is a natural contradiction with the low latency and high frequency control required by robots.
Second, current experiments mostly take place in simulation environments. The real world also increases camera noise, mechanical errors, delays, changes in friction forces and safety constraints.
Therefore, the report is closer to a capability boundary test: the extent to which a large model can advance robot control without temporarily severely limiting Token time and reasoning costs.
06
The big model is entering the robotics strategy layer, but the robot model is still irreplaceable
In the final analysis of the report, two key conclusions can be drawn.
First, the generalising capabilities of large-scale training are demonstrating a clear advantage in the task at hand.
GPT-6 Astra significantly elevated the limit of the end-to-end capability of the robot in semantic understanding, visual perception and mission planning tasks. In RoboLab, Astra achieved a 49/50 result of directly controlling the robot, showing that the general model has been able to migrate to the robotic decision-making linkage through large-scale pre-training.
This also confirms Prof. Wang's previous judgment that as the data scale expands, models can gain greater semantic understanding and visual generalization. The scaling law, which has been repeatedly verified in language and visual models, is being extended to physical intelligence.
Second, cognitive generalization is stronger and cannot automatically solve the problems of precise operations in the physical world.
In tasks that rely heavily on stability and precise control of contact, such as towers, masquerade cards, Astra Direct scores only 12 and 0. Models can understand targets but are not necessarily able to handle friction, collision, claw condition and continuous contact.
The value of π0.5 is reflected here. After adding π0.5, the score of the tower was increased from 12 to 64, the board was increased from 0 to 40, and the task of putting clothes, bottles into barrels was also significantly improved.
But π0.5 itself is also limited by the size of the data and the scope of the task. In RoboLab, π0.5 has a single zero-shot success rate of only 36%; when combined with Astra it reaches 92%, still lower than Astra Direct's 98%.
GPT-6 Astra+π0.5 is not necessarily the final form, but the report has made a path clearer: the generalized model expands the cognitive and generalized boundaries, and the physical model supplements the ability to interact with physics.
The greater the size of the data, the higher the quality, the wider the boundaries of the model's capabilities. With the further enlargement of data and computing, we may be able to really look forward to the release of intellectual value.
07
China is physically intelligent and urgently needs to continue to scale up data and computing
The pressure brought by GPT-6 Astra has left the domestic furniture industry facing a more realistic problem: OpenAI has purchased tens of millions of hours of physical data, invested over one billion yuan in data and computing resources in physical models, more than any other furniture company in China, and swordfinger hardware ChatGPT.
Under pressure, China must expand data and computing inputs. GPT-6 Astra's success rate is high, but the trajectory quality is high, the software is too complex, the model is too large, deployment is almost impossible, the cost is extremely high. This means that its route is not yet perfect.
China's manufacturing, logistics, retail and other sectors have a large number of robotic applications and have a complete hardware supply chain. But the scenario is not equal to the advantage. Only turn on-the-spot problems into data, send data into training, then put the model back into real-world verification, forming a closed loop, and the scenario can become model capability.
It is difficult to replicate every round of resource input from frontline model companies, but China has more intensive real-world application scenarios and entire industry chains. What really determines the outcome of the competition is whether these scenarios can continue to be transformed into high-quality data and then transformed into modeling capabilities.