What about the hard-trained "robot brains" of the smart corporations that GPT, Claude, Gemini are going to steal?
On 14 October, the renowned model rating agency Artificial Analysis pulled these great models into a real robotic test, where they excelled in mathematics and programming.
In the newly released AA-Robotics v0.1 evaluation, leading edge models such as GPT, Claude, Gemini, and others needed to control eight tasks such as real-world mechanical arms, waterfall challenges, writing, building blocks, and more than anyone else's "mobility".
As a result, the big models made a lot of jokes as soon as they got the job done. A mechanical arm took a dozen minutes to finally put together a few bubble blocks into a qualified tower. The GPT-6.1 Sol, commanded from behind, clearly completed the task, but still refused to stop and continue to adjust the blocks, which ended the task in failure.
This is just one of the dramatic scenes of this robot test.
The first results were equally unfavorable: GPT-6 Astra ranked first, and the general-purpose model was already able to program Agent to control the real mechanical arm; but they took an average of about half an hour to complete a task, nearly two times slower than the human baseline.
The scoreboard also provides a new basis for discussion in a recent debate in the robotics industry: What is the irreplaceable value of "robot brains" that have been specially trained by smart companies as the general model grows stronger?
Will the "robot brain" be replaced by a general model?
Last month, GPT-6 Astra controlled the mechanical arm to draw the Golden Gate Bridge, grab the bricks, and simulate human-operated video-connected brush screens. A mechanical arm, priced at around $150, under Astra's control, was able to independently plan movements and draw a Golden Gate Bridge based on camera images.
A more compelling experiment came from RoboCurve. In its real-world tests, Astra controlled two mechanical arms and put red blocks in bowl 20 times, with 19 successes. These results prompted many to ask: Since the general model already understands vision, space, reasoning, and can also write control code, why would a robot company spend so much time and money training its own VLA model from scratch?
But people in the smart industry don't fully pay the bills.
At a conference on 10 September, the founder of autonomous-variable robotics, Wang Yu, spoke of this "reduction of the impact" as unlikely to be judged at this time. He argued that the model's capabilities come from data. He argued that the programming model is strengthened, behind which there is a large amount of code data and mature verification systems; the physical world faced by robots, likewise, requires specialized data, evaluation, hardware and infrastructure, and will not be solved automatically because language models are powerful. Wang Yu also mentioned that Astra is not completely isolated from robot data.
The founder of the breakthrough robot, Xu Zheng, also experimented with Astra. It can recognize desktop objects and even grab a nut; but once it is required to perform more complex contact operations with the nut, its performance decreases significantly. In his view, advances in semantic understanding and spatial understanding are surprising, but complex physical interactions are another matter.
The focus of the debate between the two sides has become increasingly clear: whether a general model can be "handled" is no longer the only problem; how much it can do, whether it can be done steadily, quickly, is more important.
AA-Robotics just happened to offer a new measure.
GPT-6 is in first place.
But the robot test isn't that simple.
Artificial Analysis has previously been tracking the reasoning, coding ability, speed and price of large models.
Each model is controlled by a Hiwonder NexArm mechanical arm. This is a six-axis mechanical arm with five joints and a claw, with four cameras placed around the desk and on the wrist of the mechanical arm.
The model initially only knew the name of the mechanical arm, how to communicate with it through the HTTP API, and how to submit tasks. It had to look at the camera itself, understand the position of objects, write and run the program, allowing the mechanical arm to complete the movements. The GPT model ran in Codex, Claude used Claude Code, Gemini used Antigravity. All eight tests were zero-sample attempts and would not be retrained for the current topic.
The first five models have been released: GPT-6 Astra scored 69.4, GPT-6.1 Sol scored 64.6, Gemini 4 Argon scored 55.3, Claude Opus 5.5 and Claude Fable 5.1 scored 48.8 and 46.3, respectively.
However, the score is not a success rate. Each of the subjects has a total score of 100, and it is also possible to get a process score if not completely completed. Therefore, while the total score is highest, Astra gets four of the eight subjects; the second-place Sol has five instead.
The simplest of the two tasks, five models were all switched: pressing the red button with a claw, and placing a shark toy in a designated box.
In the meantime, the gap is widening rapidly.
Astra was able to put drinks in bottles into cups and write "hello world" on paper with a pen, but scored zero points in the long, wide and high measuring boxes. Sol completed the collection of blocks, shape matching and writing, but was unable to pour water.
In a computer, the model only needs to stop generating once it finds the answer; in the physical world, it must also determine whether the object in front of it has stabilized, whether the task is actually completed, and whether the next action will destroy the result.
It's even more surprising when it comes to time and cost.
Of the five models, the Gemini 4 Argon takes about 52 minutes on average, while the other four take about half an hour to complete a task. Artificial Analysis gives an artificial direct operation baseline of about 20 seconds on average. For small things like pressing the red button, Astra takes 2.2 minutes; Claude Fable 5.1 even takes about 10 minutes to understand the scene, plan the action, and start the first operation.
The average cost of a model call for each task at Astra is $16.36 and Sol at $1.93, with a score of only 4.8 points. For robots that work continuously in the home and warehouse in the future, it's one thing to do once and a hundred times a day at a reasonable cost.
It's not the same as really doing well.
Why just move a mechanical arm and the big model takes so long to grind?
Artificial Analysis dismantles the entire task process. In Astra, for example, it takes about 20% of the time to understand the environment and the mechanical arm, about 59% of the time to plan movements, and only about 13% of the actual mechanical arm movements. Sol's situation is similar: 61% of the time to plan, and only about 13% of the mechanical arm movements.
They look like a person entering a lab for the first time: first observing how the mechanical arm moves, then writing the program, testing the position, checking the image, and then issuing the next command after confirming the correctness. The action itself often takes only a few seconds, but thinking back and forth can take several minutes.
Slow is not the whole problem.
Of the six grabbing missions, the Astra's grabbing success rate was 45%, Claude Fable 5.1 was 44%, and Sol was only 28%. The model clearly found the target, but the claws may have closed next to the object; sometimes the bottle slipped from the finger.
Officially organized failed footage includes impressive scenes: the mechanical arm pushes out the toys that are already in the box in order to adjust the position of the fish toys; the wrist touches the tray when the bottle is grabbed, making the bottle more and more difficult to reach; the last part of the drink is poured out of the cup without taking into account the change in the flow position at the end of pouring water.
These errors are difficult to resolve by "thinking for a moment". In the real world, objects slip, roll, deform, and friction and collision occur when touching fingers and objects. Robots need to correct errors in a timely manner based on movement and contact feedback, rather than waiting for the big model to redesign every move.
This is also where professional body models still have room to play. VLAs, underlying movement strategies and long-term controllers, are how to quickly transform vision and mission objectives into continuous, steady movements.
Of course, this list does not directly compare General Agent to Special VLA under the same set of conditions. Eight desktop titles cannot represent all robot tasks, let alone hand-to-hand collaboration, flexible object operation and long-term autonomous work.
But it's at least one step ahead of the debate: General Models can do real operations, there's already systemic evidence; how far away they are from doing real work efficiently and reliably, and for the first time in this group of tasks, they've been measured specifically.
In the first list of AA-Robotics v0.1, there is no Chinese model yet.
So will the next batch be Qwen, Kimi, GLM, or DeepSeek? If we put the same mechanical arm, the same task, can domestic models keep up with or even outperform these top international models at a lower cost?
This is perhaps more to be expected than simply competing for a list. The next phase of the robotics industry is not just about "who can get the machine arm moving", but also about who can get it moving a little faster, get a little more focused, and know when to stop after success.
The link is as follows:
© THE