One presses down 21,000 GPUs per hour, one presses down one skill training to 10 hours.
On 12 August, the journal Science Robotics published two papers on the control of robotic movement in the same journal.
One is the ZEST, a zero-sample migration exercise simulation framework developed by the RAI Institute and the Boston Dynamics Joint Team, which is featured on the cover of this issue.
Another from the Invidia GEAR team built a robot movement tracking base model calledSONIC.
The two sides are staring at the same question:How can robots become more and more familiar with human body movements?
But the way we spend money and energy on both sides almost goes to two extremes.
ZEST is a lighter route.
It integrates action capture, ordinary single-eyed video, and even critical animation into the same training process. For specific skills, an NVIDIA L4 GPU trains for about 10 hours, deploying the strategy directly from simulation to real-time.
Invidia moved the Scaling route familiar to the Big Model era into the control of humanoid robots.
SONIC consumed about 700 hours of motion capture data, over 100 million tons, and expanded the model parameters from 1.2 million to 42 million, ultimately using 128 GPUs to train for seven days, for a total of about 210,000 GPU hours.
On the G1 real-time machine,SONIC tested a sequence of movements of 123 groups, with a success rate of 99.2%, and supported real-time tracking of human movements.
One is 1 GPU, about 10 hours, for training efficiency and algorithm design; the other is 128 GPUs, 7 days, for data, model and computational scale.
In other words, ZEST is lowering the marginal cost of a new skill set; SONIC is willing to pay higher one-off training costs in exchange for greater coverage.
01.
It's like teaching robotic moves, and counting two bills on both sides.
Today's humanoid robots are no longer lacking in demonstrations.
The problem is that it can't bend down, kneel, roll, dance, drill through low space; it can't use its knees, arms, or even thumbs to support when it hits a box; and it doesn't have to redesign a control logic after another move.
Traditional model controls are good at handling well-defined movements, but the complexity of the movements causes the amount of work to expand rapidly.
In particular, many body-contacting movements such as dancing, rocking, rolling, when robots use their feet, when their hands are on the ground, when their knees are on the ground, contact sequence, friction, impact, and dynamic errors are all overlapping.
ZEST's research team specifically notes that traditional full-body MPCs typically revolve around a predefined end-to-end contact design, making it more difficult to deal with movements involving any part of the body such as the knee, forearm, and forearm.
Reinforcement learning bypasses some of the obvious models, but it also brings new problems.
Each type of behavior often involves new reward designs, training processes, and parametric adjustments. The team also sees this as an important reason for the difficulty of scaling man-made controls: the reward function of walking, which does not allow natural church robots to dance, crawl, or complete remote operations.
So the two papers were chosen separately from the two cost ends.
ZEST compression is the process by which hydrogen produces a new skill tank. Sonic compression is the need for hydrogen to train a controller tank separately for each skill.
02.
ZEST: A new movement to squeeze the church into a day
ZEST first made the input to the action data very broad.
Professional motion capture can be used, ordinary single-shot videos taken by mobile phones can be used, and even critical animation that does not fully conform to real physical laws can be used as a reference.
These data ultimately go through the same pipeline: recording → three-dimensional reconstruction → action redirection → simulation training → actual machine execution.
The most realistic of these is mobile video.
The team used a handheld cell phone to directly capture a person's movements, retrieving 3D postures from the video, and then mapping human movements to the robot's body. According to the team, converting a good shot into a robot reference movement takes only a few minutes, so the actual process can be done: recording a movement in the morning, training during the day, and running on a camera at night.
ZEST uses two key designs to teach complex movements.
The system cuts long moves into multiple segments, where there are more failures and subsequent training takes more samples, rather than spreading the calculation average out.
Another is a bit like learning to ride a bicycle. Early in training for high-dynamic movements, the system imposes a virtual auxiliary force on the robot to help it get through the most likely to fall; as the movement matures, the auxiliary force will gradually decrease to zero. In a meltdown experiment, after removing this auxiliary course, the performance of 20 hours of training is roughly equal to the level of 10 hours of ZEST training.
As a result, a relatively streamlined approach has been applied to three very different types of robots.
Atlas completes walking, running, rolling, jumping, folding and dancing; G1 completes ballet, jumping boxes, and climbing boxes; and Spot learns from animation to stand up, roll barrel, and fold continuously.
It should be noted, however, that ZEST's zero-shot rule mainly refers to the fact that once training in simulation is complete, the strategy is no longer applied to the real machine, and Sim-to-Real deployments are directly performed, which does not mean that the robot can immediately learn to see a new action that has never been trained. In real machine experiments, each skill still corresponds to a specially trained strategy.
So ZEST, at its current stage, is a more precise positioning, a high-efficiency skill production line.
Every time a robot learns a move, it still needs to be trained, but training the artificial design and engineering steps it needs is compressed as much as possible.
03.
SONIC: Pioneered 21,000 GPUs per hour and practiced a set of general-purpose gymnastics
In the meantime, we're going to have to make a decision.
Since language models and visual models have shown that the ever-expanding data, models and computing power can be exchanged for better generalization, can humanoid robot controls also scale?
This is where SONIC starts.
The team defines human-robot tracking movements as a basic task: to give the robot a reference to human movements, so that it reproduces as much as possible in compliance with real physical constraints.
Then directly extend the three variables.
Motion data expanded from small-scale to 700 hours, over 100 million tons; network size expanded from 1.2 million parameters to 42 million parameters; training computation from about 2,000 GPU hours, 9,000 GPU hours, to 21,000 GPU hours, with the largest version using 128 GPUs to train for seven days.
The result is not just that the model improves the training.
In the action categories that did not appear in the training set at all, the largest model simulation tracking success rate was 99.6%; after deployment to the real Utrecht G1, the team tested 123 different movement sequences with a 99.2% success rate.
There is a very different goal here than ZEST.
ZEST's logic is to train a skill and a reliable strategy efficiently. SONIC hopes to train a broad enough strategy for the next moves to fit into the sporting space it has learned.
So the really important part of SONIC is still behind us.
In addition to this general strategy, INVEDA has created a unified interface.
The camera captures human gestures in real time, allowing the G1 to follow human movements; VR devices can be operated remotely; text, music, and video are generated by upstream motion models into target movements and then handed over to SONIC to execute.
Single general control strategy is driven by multiple input methods. A) Video remote operation: estimate whole body movements from a single RGB video stream and track them in real time. B) Text control: The target movement is generated by natural language cues. C) Music control: The target movement is generated by music and adjusted according to melody and rhythm structure. D) VR full body remote operation: The operator controls the robot to complete a series of dynamic whole body movements. Each panel displays a sequence arranged in chronological order.
There is a distinction to be made:SONIC itself is not a language model that directly understands text, but takes on a physical execution layer.
On the upper level, Intel brought in the GR00T N1.5 VLA model.
VLA is responsible for predicting movements based on visual and mission objectives, and SONIC's Unified Movement Token is responsible for transforming these goals into full-body movements.
VLA-powered mobile control tasks. Each row shows a sequence of steps arranged in chronological order. Time from left to right. A) Apples are taken to the plate via a three-point interface. B) Carrots are taken. C) Brushes are taken. D) Tramples are opened.
Thus, the G1 can not only go to the table with the apple, but also to the garbage can, opening the lid with one foot on a pedal, while maintaining a single-legged balance; in the more complex task, it must first pick up the drawer, then move to the garbage can, step on the lid, and finally throw the lid in.
The VLA+SONIC system has an average success rate of 75% among the five categories of mobile operations.
04.
What kind of body shell does a robot need?
Here's a look at the most eye-catching figures from the group:
EST, a specific skill strategy, uses a single NVIDIA L4 to train for about 10 hours; Sonic uses 128 GPUs to train for seven consecutive days.
There is no direct comparison between the two.
Since ZEST is still a training unit with specific skills, solving is how to move a new movement faster, less engineeredly to the real machine; while SONIC tries to squeeze a large number of movements into a unified strategy, solving is how to make the same controller itself cover more movements.
A man who hopes to make a new move will soon learn to make a move; a man who hopes to make a new move will do well to make a move.
But in the whole robotic system, the solution to both is still the same layer of problems.How the body moves.
On top of them, there are VLAs, Agents, mission planning and motion generation models, responsible for answering what robots are going to do now; down the line, there are joints, motors, and real robots themselves, responsible for actually executing the movements.
This also means that ZEST and Sonic are just two engineering options for robots to gain full body mobility, not necessarily leading to a true binary option.
Whether or not it is possible to quickly generate new skills at a low cost for a relatively clear but increasingly varied scenario of action is important; and a general motion controller with sufficient coverage and repeatable calls is becoming increasingly important as the upper-level VLA begins to shift its larger movement target towards robots.
In the long run, the two ideas may even come closer.
ZEST has attempted to integrate 15 actions into a multi-skill strategy in simulation; while SONIC has expanded its general action coverage, the control interface is further connected to real-time human gestures, VR and VLA.
The routes are different, but the direction is becoming clearer: robots can't always be carved out by engineers from one movement to another.
The real solution to the next stage is how to continue to expand physical skills as much as model capabilities, which can be both pre-trained on a large scale and quickly supplemented when new skills are encountered.