“ There is still no definite direction for physical intelligence, and we are all on our way.
At a recent roundtable, six experts put on the table some of the most central technological challenges in physical intelligence today. Huang, Director of the Center for Artificial Robotics at the Beijing General Institute for Artificial Intelligence, focuses on the full-body synchronous operation of two-legged humanoid robots; the lead scientist of the StarHeart Guide on the training model for focused physical models; an assistant professor at the Shenzhen International Graduate School of Technology at Qingdao University in Shenzhen who promotes the fusion of RLinf and Agentic Robotics.
The whole discussion proceeded along three lines. The first was the model layer, the VLA is the most reasonable technological route at the moment, the world model is the real problem or the buzzword. The second was the data layer, the fact that one million hours of human video can't be replaced by a robot's scaling law, how to use multimodal and causal signals more efficiently.
This article organized this roundtable conversation to impress readers.
Host Chen: It's a very exciting time, in Silicon Valley, where tech giants are working hard to build a team of robots, and many startups are working hard to build models and make new releases. Robots can do all kinds of interesting things and land applications right away. But beyond the excitement, I think the industry should cool down, and robots are just getting started.
I am now a researcher at the Center for Body Robotics at the Beijing General Institute of Artificial Intelligence and co-founder of Deta Intelligent. We are focused on the basic model of two-legged humanoid robots, and want to use this basic model to solve the problems of local manipulation (local operation) and robot-to-body synchronization.
I think the most interesting thing is that we've learned a lot of generalizations from human videos, but one thing we're still not sure about is that the generalizations we've learned from human videos, are they visual generalizations, action generalizations, or semantic generalizations? Or have I just aligned multiple models of visual, semantic, and so on? I think it's worth continuing to explore.
I am a graduate student from Shenzhen International University of China. My own research focuses mainly on reinforced learning-driven physical intelligence, focusing on the collaboration of algorithms and systems. In recent years, our team has been working on a large-scale reinforcement learning framework for physical intelligence, called RLinf, which has also gained some recognition in the community.
I'm the CEO of Current Robotics. We're mainly focused on local manipulation algorithms. What we're most concerned about is how to build data from all the local manipulation to the basic model, to the ultimate world simulator, to be able to turn it all into a loop. Including the recent special fire of RSI (Recursive Self-Improvement), I think the future has a complete infra that can take this to a new level of Physical RSI.
I'm an assistant professor of research at Northeastern Institute of Artificial Intelligence. I mainly work in the direction of causal inference and artificial intelligence, and recently I've done some work focusing on the combination of cause and effect. What we're concerned about is the shift from relevancy to causal paradigms.
I'm the co-founder of Dyna Robotics. One of the most important things we've done this year is just recently launching massive pre-training, pre-training with human video. Our main motivation is that artificial data is the most scalable data. The entire industry wants to replicate the success of big models, the first thing to solve is how to find big model-like data sources that can continue to generate data, and we can make good use of it. But at the same time, we also focus on how to really use these basic models in real-world workplaces.
RLINF, RSI, causal relationships, models... these are very important topics, and we're going to go through a series of questions slowly.
First of all, let's talk about a question about the model. First of all, let's go ahead and ask a few questions, because G0.5 was recently released on Stereotypes, and I see you've made a number of innovations in the model, including a unified asymmetric action decoder, a real-world action four-dimensional engine time and space attention module.
01.
Q1: Why did you choose these three directions to upgrade and what do you think is the next step for VLA?
A: First of all, I don't think VLA is necessarily the right direction, but we think it's a very reasonable technological route at the moment. The main reason is that many people have discussed the importance of language in the past.
I often give examples of why we don't get monkeys and orangutans to work in our factories? Because these animals don't fully understand human intentions. If I'm going to teach an orangutan to do a task, I probably can only teach it by learning to imitate, and learning to imitate is a very inefficient way of communicating knowledge.
If we only had two models of vision and action, we would probably only be able to follow the path of real machine reinforcement learning. But what if we couldn't do real machine reinforcement in the pre-training phase? Introducing language is actually a reasonable way to really align ourselves with the intent of the task, how to complete it, what are the risks, what should be done.
So we made a native visual language action model. What we call native is that we put all the main parameters of visual, language, and action in the backbone of the VLM, that is, the Language model, without an Action expert. Most VLAs in the past had Action experts, and we didn't have them.
What we do is we dissociate the action and put it into the language model. In the process, the dissociated action token is basically a foreign language to the language model, as the 201st new language to emerge in the world, but it is closely related to the languages of other countries in the past.
In fact, the VLA model is very much like this: action is actually the last translation of visual observation and language, which translates into the physical world and the interaction of the physical world. This is the motivation behind our design of the VLA model, unified parameters, complete decentralization, self-return. I think this is a very important technical route, not to say that it is the end, but it is very important. So we explored and invested a lot of resources.
Host: In addition to the VLA, there's a keyword in the model this year: world model. I've also heard many critics in the industry say that world model is a buzzword.
02.
Q2: Is the world model really a buzzword? If not, in what ways does it really help train robots? If so, why are so many companies deploying world models now?
A: I think it's half-and-half -- some people might dislike the term buzzword because, at some level of publicity, it may be that the model architecture or some of the details in the design are exaggerated to reflect the output results. But from the logic and principles of the design, I think the world model is essentially an improvement to VLA's prediction of the state, which is crucial.
Because even for more closed-loop, longer-range actions, it may be necessary to predict what will happen in the future, including visual and linguistic situations, with more precise predictions. So a future world model is equivalent to providing a basis support for such a prediction.
In actual use, we've also found that it can more efficiently leverage some of the video and multimodal distributions in the data, allowing the model to not only measure a series of movements through agent code, but also to make the movements more closed-loop, longer-range tasks, and more efficient use of large volumes of video data.
Now that we're talking about scaling, we want to do 100,000 hours, like DYNA is a million hours. It's crucial how efficiently we use this video data, how efficiently we predict the environment. In this respect, the world model has advantages over VLA, but the buzzword part is some over interpretation and the essence is about how we build better data to train both models.
A: The reason why different people understand the world model differently today is because different industries and research backgrounds are different in different industries, different people see the term differently. I used to do reinforcement learning, so starting from this perspective, the world model in our training diagrams or data diagrams represents different roles and different roles.
In our concepts, world models have two uses. The first use, reinforcement learning, is to interact with the environment. Today's environment is either a simulation or a real machine.
On this level, the role of the world model is to provide very realistic, very good observation on the one hand, but also to provide very good dynamic simulation or physical parameter simulation, so that people can simulate the physical laws of the world. So many people would like the world model to be able to do more realistic image rendering, to simulate the whole world very well.
The second use is that people today understand it from the perspective of reinforcement learning or robotics and want the world model to be a representation. If there is such a representation, I can infuse unit data, human data, or even asymmetric data into the world model at the same time, and then it no longer needs real image rendering, it actually only needs to do a unified embedding.
So today, when you look at the reports or listen to the discussions, no matter what, people have different technological beliefs. In essence, everyone wants the world model to function and function differently, which makes the information look very redundant.
Congratulations: I agree with the teacher's view that the World model is a buzzword because it can be big, it can be very universal, it can be very small, it can be done in different applications.
For the robot world model, it is now mainly divided into two parts. One is that everyone is doing very good WAM, including teachers who are working on the robot; the other is the simulator that superteachers are talking about.
The most critical part of a simulator, besides doing data scaling, doing model scaling, is evaluation scaling. I believe that in the future there may be companies that specialize in evaluation, and we need very broad benchmarks to evaluate different models efficiently, fairly, and iteratively, to see if it really achieves better capabilities.
The Simulator will have great advantages, the first being that it needs to have two points.
The first is controllability, which requires good control over the robot's movements; the second is interactivity. Starting from both perspectives, using it as a predictor to see if the model's strategy is feasible, and then take over as a post-training in it, these types of applications I feel are already in place in previous areas and are generating value in the cycle of our daily iterative models.
03.
Q3: How do you see the world model now in Silicon Valley?
In Silicon Valley, the world model is also a very new trend, as most companies are doing. But as the previous teacher mentioned, the definition of the term "world model" has become very widespread, such as WAM is more characterizing, then simulator; there are some who do 3D, jump from the form of video to 2D to make world models. In fact, I think the role of world models in different industries, in different fields is really different.
PRESIDENT: Next I want to focus on data, because this is also a very core industry issue.
04.
Q4: York, you just released DYNA-2 at the beginning of August. According to the disclosure, DYNA-2 uses over 1 million hours of egocentric first-person human video to pre-train and adapt to specific robots with a small amount of local data.
A: We feel that at least we have seen the possibility that scaling law could happen. In the past, when we trained real machine data, the biggest problem was that the amount of real machine data was very limited.
Last year (in 2025) we spent almost a year trying to do scaling law with real-machine data, but it didn't work out so well. It's mainly possible to achieve high accuracy or success rates in some vertical applications, but there's a big problem with generalization, where robots can't understand these changes when the scenarios change, or when the objects they're working on themselves change significantly.
So from last year onwards, we've been turning to more extensive data -- and we know more clearly that this isn't something that you can do with money or your heart, and that some of the data is going to be very slow to generate.
We've seen some other forms in the middle, and we've tried them. But they end up being more biased towards the middle form, requiring some equipment when it comes to data collection, and the equipment itself is very troublesome for scaling. Whether you're new to the device or you're training people to understand and use the device. So we end up choosing more human video because human video doesn't need to be specifically operational.
The core goal of DYNA-2 is to verify what is known as "human data, can it really be transformed into a robot". The main reason why people didn't believe in human data was because of the huge differences in the current model of robots, and the variety of robots, which is a very difficult gap.
In the long run, we thought robots would be at a very popular stage, so we tried DYNA-2 on this premise.
When we put all the human video data into the model, while the human video data is increasing, the accuracy of robots on the test set is decreasing instead. This is a very shocking point, and it's very difficult to show that human video information really translates to the ultimate very precise downstream accuracy.
PRESIDENT: There are actually three different data routes that are completely suppressed by different companies with data issues, for example, we see Core Robotics, your CoreZero is using the whole-body and out-of-body data collection system as your route; I see 21,000 hours of human data in your public data, of which 28,000 hours are whole-body registration, whole-body action capture.
05.
Q5: Why do you take this data route?
A: I think, as York just mentioned, whether it's ego-centric data or UMI data, it's essentially human-centric data, it's just a different way of implementing it. It could be in a more lightweight way, like wearing a camera; it could be in the middle of wearing gloves, including UMI's claws.
1. Data diversity. To have enough diversity at the visual level, large-scale collection in reward is required.
2. Action diversity. Action should be accurate enough to avoid wasting a lot of resources on labeling and GPU training.
So we did this out-of-the-box program, and we wanted to take this very difficult path in local manipulation + agility, to be able to take the first step in collecting data that is "scaled, high-precision, visual diversity" and at the same time be able to collect it at a low cost and drop this route of teleoperation to build our own model capabilities.
Host: All in all, what are the data routes on the Star Haitian map?
In our judgment, big-language models have a limited amount of data, but there is no limit to the amount of physical data. Although it is rare now, in the future, the entire industry and the government are pushing for data collection, so that in the future, data from both virtual and wearable devices will slowly increase. We are not particularly worried about data collection after robots, which is a basic idea.
PRESIDENT: I'd like to hear from the other three guests some of your thoughts on data.
From the first principle, robots should make the best use of human data as much as possible. York has just said that to reduce the environmental guard as much as possible, if there is an environment guard, in the process of data processing, retargeting will lose a lot of information, while making the most common data model difficult for robots to learn in many situations, inventory will lead to a lack of long-tail distribution.
So we think that doing data should preserve data that humans can learn as much as possible. Our program is a way of capturing the ego from the first point of view + the whole body so that when people operate, they can not only see the movements of the five fingers, but also see the whole body, including the lower limbs. The egocentric data recipe that we see now is actually missing the lower half of the human body, or a huge amount of data registration with the whole body.
A: The teachers above talk more about pre-training data, and I myself, because of the differences in research directions, will pay more attention to the data required after training than the data from pre-training.
For post-training, the focus is on what the model "doesn't" data is. For a more understandable example, pre-training is shaping a boundary of ability; the goal of post-training is to extend that boundary outward.
1. To find out exactly where the boundary formed by pre-training is, this is my verification period, to find it first "where it won't";
2. To expand further, how to supplement where it will not.
Both of these things are essentially about finding "the data you don't", which is more of a long-tail distribution. Another is that after a robot is deployed, it requires continuous post-training, which is a part of the data that is more difficult to obtain: it takes a lot of redundant information to get rid of, it takes more unique data to get the model to achieve a 100% success rate in the long run.
So when it comes to the academic world, it's more about looking at what our research is going to be two or three years from now, and what our research is going to be about, and what we're going to be talking about in the industry today, it's really important to talk about data and to diversify pre-trained data.
A: Whether it's pre-training or post-training, because I'm an AI scientist, I want to say: although we emphasize that pre-training should be done better, diversity is high in the data, if we look carefully, there will be some false correlations, regardless of how we do the math.
For example, some of the actual algorithms and products that we've seen in person today are vulnerable to the effects of changes in desktop texture and color in the process of testing, especially capturing this task. I think the first is that we don't get enough data, but the second is that data can't be collected alone, and we can't actually get enough data across all possible environments, and not only to get through, but to be consistent enough that it's the best theoretically, but it's actually hard to get it in practice.
I do cause, cause is very important, it won both the Turing Prize and the Nobel Prize in Economics in 2021 for the scholar of causation. Causation is primarily the first correlation, and this is a bottleneck that exists now with the World Action Model, the transition between states that we understand, are they often occurring together, or is there really an action that can lead to a transition from one state to another?
I'll give you another very simple example: as a child grows older, the size of the shoes he wears increases, and the mathematical knowledge increases. So if we just look back at these two variables, we'll see that the bigger the shoes, the stronger the mathematical knowledge, that's just correlation.
PRESIDENT: There's another question I want to ask the professor, you're doing reinforcement learning, and reinforcement learning has been hotly discussed this year in this piece of robotics.
06.
Q6: What progress has been made in real-machine reinforcement learning RL this year? How far to scale up yet? What are the main challenges facing now?
A: It's a good question. Our team hasn't started any new work since the year 2026, mainly because we found some bottles inside ourselves. Most people in the industry are now doing real machine reinforcement learning on a single robot.
1. Number of robots/multi-machine stability. Even today's manufacturing power in China, everything takes time to develop until it slowly matures.
2. Today's pre-training relies heavily on man, for example pre-training relies on man to find human data, to have a camera on his head to record video of a human hand, which essentially requires involving a person relatively lightly.
So I think today's real machine reinforcement training, the first is that we're mostly limited to single machines; if it's multi-machine, we're all limited to offline. Our real goal for reinforcement learning is to be online, large scale, or we're exploring how to make it happen.
Host: The next question is the RSI.
RSI is a very important keyword in the big language model LLM this year. The robotics industry also has its own RSI, a loop of AI evolution that breaks down human relationships to form an AI self-reinforcing AI loop.
07.
Q7: In the robotics industry, has the RSI closed loop been formed? For example: deployment generates real machine data → reward model automatically generates data → RL / post-training → world model synthesis and evaluation → next generation deployment.
If you say "self", I don't think that's true, because the parallels are too small, not very true. So here "self" is actually talking about a robotics or multi-machine industry. I sometimes feel that the word "self" may have different meanings in the robotics industry, but if you let me define it, I probably wouldn't use that word much, but I think it still exists if we just say "data automation within the big industry".
RSI for Robotics is actually a very interesting problem, and it also incorporates some of the agentic pipelines we're talking about right now.
Here are a few from the real robot learning process:
1. The problem with Verify is that the RSI relies heavily on verify, i.e. how to ultimately verify the success of a task. It is relatively simple in a restricted environment, but general robots usually deal with some of the more open environments. In an open environment, how to design Verify, and how robust it is, is a big problem. If not reliable, the model can do very serious reward hacking or model collapse, and the model is trim bad.
The problem with Sim 2 Real is that we've just talked about recursive or model work that can be done with simulation, but that involves a lot of sim 2 real gaps. But simulation overlaps faster, and eventually we have to deal with the bigger sim 2 real gap and the scalability of simulation.
So I think in a limited environment, RSI for Robotics is interesting, and we've seen some good results. But I'm more concerned about how to open up the world to what I've just said is scalable, including how it's scalable to multi-machine processing, and how to better combine verify with environmental feedback and turn it into robot grading so it can learn.
Congratulations: I think the RSI should be def first