It is now a clear consensus that the narrative of being physically intelligent is shifting from "doing one action" to "being able to accomplish complex tasks steadily in the real world". Especially in the recent WAIC 2026 scene, it is clear that robots are no longer satisfied with performance-style single-action presentations, but are expected to enter production lines, warehouses and hotel rooms to truly "do the work".
However, there is a clear gap between the demo and the landing. VLA or WAM? the model isn't big enough? the parameters aren't large enough? etc. The route is uncertain.
Earlier, AI teacher Lee Feifei, professor at Stanford University, reiterated this difficulty, saying that unlike LLM, which has access to a wealth of data resources from the Internet, physical intelligence from training to evaluation, data is scarce...
Look closely at how many big data sources in the industry are currently doing well.
Internet video is large in size, but lacks precise action information; real-time remote data is accurate enough that it takes robots, operators, and fixed locations at the same time, costs are high, and the expansion is slow; non-native collection programs such as UMI offer another path to dismantle the collection equipment from robots, solving the problem of getting into more real environments.
But over a long period of time, this type of inorganic data, due to its lack of reliability, was mainly used for model pre-training, to the post-training stage of learning specific tasks, and the team still had to supplement a certain proportion of real-world remote-operation data, as "focus points", to align the robot's core, execute time delays, camera angles and real-world differences.
It is common practice in the industry to reduce the share of remote data in this part of the machine, reducing the "focus point", but as long as the "focus point" remains, it means that no physical data can always be completed independently from training to deployment.
Is there another possibility: instead of continuing to compress real-time data, can we improve the reliability of the non-material data itself and lose the need for "focus points"?
This is the starting point for the introduction of HiFi-UMI by Shenzhen Smart.
Recently, Shenzhen Smart has released a high-performance, unobtrusive HiFi-UMI data production engine that connects portable collection equipment, trajectory reconstruction, data governance, model training and machine deployment.
In the experiment, the post-training phase used only HiFi-UMI demonstrations to deploy the strategy to real robots. In the three representative models of both VLA and WAM architectures, the overall success rate reached a relative level of the same real-domain remote-operation data. At the same time, the model further decreased the average error in predicting movements on ten unseen missions by 41% after large-scale pre-training using the same set of contents.
This means that non-native data starts from a data source used to expand pre-training content, moving on to post-task training and real robot execution...
Link to the paper:https://arxiv.org/pdf/2607.25895
The main page of the project:
Open source data sets:https://huggingface.co/datasets/simple-world-lab/HiFi-UMI-2K
"Details without substance" - why is it that you stop at the pre-training stage?
In fact, for a long time, UMI's data has been trained "out of doors", not with industry bias, but with real technical limitations.
The first thing to admit is that UMI does solve the most intuitive scale problem of robot-operated data.
Real remote operation requires robots, operators, and mission environments to be online at the same time, and after each failure, objects need to be reset, robots need to be restored to their original state, and hardware may also experience wear and tear. With scale expansion, equipment scheduling, personnel management, and site maintenance will become new costs.
While the UMI-class uninhabited collection disassembles human samples and robot hardware, removes the collection equipment from the robot, without the need to manipulate a real robot, and only carries portable equipment into homes, hotels and other scenes, to record the human operation of wiping, sorting, folding, and transportation, thus getting rid of the limitations on the number of robots and entering more real environments.
But at scale, it doesn't mean that data can directly drive robots. Because of pre- and post-training, the requirements for data are not the same.
In the pre-training phase, the model can learn from the data which object the hand should approach, which steps a task typically contains, and roughly in which direction it should move after grabbing. In the post-training phase, the requirements are significantly improved, the robot needs to know where the end should land in a specific position, what distance and angle both hands should maintain, which claw should close, and how to continue to move after contact.
Millimeter-level errors do not prevent the model from understanding "putting the remote control into the slot" but are sufficient to cause "real insertion" failure.
So the core challenge is that data is not reliable enough from pre-training to post-training.
Traditional UMI, on the other hand, has four distinct limitations on reliability:
Trajectory drift: The position depends on SLAM online vision of the wrist, the perspective is frequently obscured by the hand or object during the operation, and trajectory drift problems are severe in long-range tasks;
Both hands "speak differently": the relative position of the two hands is reconstructed through a cross-camera view rather than native measurement, coordinating missions errors;
Time discrepancy: cameras, claw encoders and location sensors often come from different devices, relying on software or wireless alignment, in which the model may match the visual state with the wrong movements, and there is a millisecond time lag, which directly affects the learning strategy at the moment of fast movement, claw closure and contact with objects;
Visual-blind zone: only one camera with a fish eye, the visual-blind zone is large, the depth of the clue is weak, the model sees the beginning and the end of the movement, but it is not necessary to see what is happening in the middle.
These problems together limit the limitations of traditional UMI data capabilities: it can help models form a general operating experience, but it is difficult to independently undertake training on tasks that require higher accuracy.
HiFi-UMI software and hardware collaboration, "quadruple authentication"
Shen Park Intelligence believes that since reliability is insufficient, there is a way to get rid of this "hard bone". For this reason, Shen Park Intelligence did not make repairs based on the old process, but around the four dimensions of "trajectory accuracy, two-hand positioning, time synchronization, field of view coverage", and systematically restructured the collection of hardware and data processing.
SLAM with double-eyed inertia, solving the relative positions of the natives
Traditional UMI typically places the main perspective of visual positioning on the wrist, while HiFi-UMI moves the positioning benchmark to a relatively stable head, using a head-mounted dual-eye camera and an inertial measurement unit (IMU) to reconstruct head trajectory via offline dual-eye inertial SLAM.
At the same time, the left hand installed the visual marking structure separately, positioned simultaneously in a unified coordinate system by the head double-eyed camera, so that the relative positions of the two hands were formed directly during the collection process, without the need to rebuild the two tracks separately, and then collage afterwards.
As a result, in a handwritten trajectory experiment, HiFi-UMI was able to reconstruct a millimeter-scale pinpoint motion, with an end-to-end trajectory error of about 3 millimeters in the workspace, and was entirely independent of external motion capture or positioning equipment.
Time-level alignment: Unified GPIO hardware synchronization, eliminating cross-sensor bias
The traditional multi-second software is synchronized, where HiFi-UMI uses a unified GPIO hardware trigger to replace a six-way camera, IMU, and claw encoder that is driven by the same "neural" and compresses the synchronization time between sensors to less than 40 microseconds.
The robot strategy is to learn what actions should be performed when a certain vision and state is entered. If the image, hand posture and clavicle state are not strictly synchronized, the state and action in the training sample will be incorrect. Micro-second hardware synchronizes the vision, trajectory and clavicle state to match the same physical moment as possible.
6 Road Vision Synergy: Complementing global positioning and local perception
In terms of visual coverage, HiFi-UMI has two ultra-wide-angle fish-eye cameras configured on each hand, plus head binoculars, forming a total of six-way visual input.
The head view provides comprehensive global observation, recording the environment, target objects, and mission progress; the wrist view is close to the claw and operation area, responsible for capturing touch, grab, and object localization changes. The single-wrist and vertical view are about 200°, while the other viewpoints may retain critical operational information when the target object is obscured in a video path. The system can identify operational objects, determine mission progress, determine movement boundaries, and provide more basis for trajectory checks, failure analysis, and AI-assisted markup.
It's a handful of gloves to make it more "human".
In addition, it is worth noting that HiFi-UMI also uses a full-sleeve glove claw, combining fine-grained handling of small objects and grabbing of larger objects. Collectors can complete the demonstration using approaches closer to natural grabbing and force, reducing the interference of the equipment structure to the habits of movement.
However, the collection is just beginning, and high-precision equipment can only ensure the quality of the original signal, and once a human specimen becomes model-available data, a complete data governance process is required.
Once the data collection is complete, steps such as track rebuilding and cleaning, simulation redirection, AI-assisted marking, artificial verification and analytical output are followed.
The pass rate of track reconstruction and simulation relay is about 98%, and simulation relay can check in advance whether the action exceeds the robot's workspace, whether the two arms collide, whether the claws are in good condition, and whether the entire operation can be executed continuously, etc. The basic efficiency of the original data collection is about 96% after the two links are linked.
Eventually, each effective demonstration will retain synchronous multi-visual video, two-handed tracks, claw states, language descriptions, submission boundaries and quality records.
Here, HiFi-UMI has officially evolved from a collection of devices into a complete data production engine.
The real test: Is it possible to "go straight to the real machine"?
Data is more accurate, and only solves the problem of data production. The real test is whether robots can complete tasks with this data alone. That is, in the post-training phase of the target task, completely without supplementing the real machine remote data, can the trained strategy complete tasks on real robots?
To answer this question, the team set up four missions on the two-armed virtual machine platform.
Dirty scrubbing: robots need to keep clean tools in contact with the desktop and cover the target area according to a reasonable trajectory. Such tasks test continuous contact, movement coherence and trajectory control. Dirty robots may not be able to maintain stable contact and cover the entire area even if they can grab tools;
T-shirt folding: flexible objects do not have a fixed shape, each step the robot completes changes in the ribs, edges and position of the clothing, the two hands must also cooperate with each other, maintain tension, complete alignment and folding, and the relative position and long-distance motion stability of the robot hands are tested;
Remote-controller insertion: This is a typical constraint placement task, where the robot needs to accurately determine the position and direction of the slot and adjust the remote controller to the appropriate posture, a few millimetres of misalignment or a slight angle deviation, all of which can lead to the failure of the insertion;
Classification of fruits and vegetables: The robot needs to use its right hand to place the vegetables in the pink plate and its left hand to place the fruits in the blue plate, while testing the instructions to follow, visually understand, and handle the arms.
Overall, the setup of the four tasks involved "continuous contact, flexible object operation, precise placement and instruction-following" respectively, avoiding experimental results from simply grabbing or moving fixed objects.
The experiment selected three representative models: the StarVLA-QwenPI, the OpenPI-π0.5 ton VLA model, and the LingBot-VA ton WAM model.
The three models use different technical routes to separate the data source effects from the model architecture effects, so that the validation of HiFi-UMI is no longer limited to a specifically tailored strategic architecture.
In the same model, the experiments only replace training data sources, one group uses real-machine remote data, and the other group uses only HiFi-UMI samples.
It should be noted that the scenarios facing the two types of data are not the same. The remote-operated baseline of the machine directly collects from the final evaluation environment. The model has seen the same desktop, background, lighting, camera viewpoints and object distribution in training. While the HiFi-UMI samples come from different locations, backgrounds and lighting, there is no single HiFi-UMI trajectory collected in the final evaluation field.
Thus, it can be said that the real-time remote-operation baseline therefore has a clear advantage in the same-domain data, and the HiFi-UMI strategy requires, in addition to learning task movements, to transcend additional environmental and visual distribution differences.
However, from the results, even under such "unequal" conditions, HiFi-UMI data still performs well.
Below is a comparison of HiFi-UMI and real-time remote-operation data on two VLA models, with 40 real-time evaluations for each task and a combined total of 160 real-time evaluations for each strategy.
The overall success rate of StarVLA-QwenPI using both types of data was 51.3% and 53.8%, respectively, i.e. using HiFi-UMI post-training, 82 out of 160 ratings, and using remote-operated post-training, 86 out of 160 ratings, with only 2.5% difference.
While OpenPI-π0.5 has an overall success rate of 77.5% and 74.4% for training based on HiFi-UMI and real-time remote data, HiFi-UMI training is 3.1% higher than remote training, respectively.
Similarly, on the WAM model LingBot-VA, the overall success rate of post-training HiFi-UMI and post-machine remote training was 56.9% and 57.5%, respectively, with a gap of only -0.6 percentage points; while on scrubbing and vegetable sorting tasks, post-training HiFi-UMI achieved a success rate of 62.5% and 57.5%, respectively, slightly higher than post-machine remote training.
In fact, the key to the results of this group of experiments is not just the success rate numbers themselves, but the fact that high fidelity non-material data can provide mission meaning, visual observation, precise trajectory, hand-to-hand relationship and execution timing at the same time.
In other words, HiFi-UMI began to have the ability to independently undertake targeted post-training tasks.
In addition to replacing real-time post-training data, HiFi-UMI can also be used as large-scale pre-training data.
The team used more than 4,000 hours of HiFi-UMI data to train on StarVLA-QwenPI.
The results showed that on 10 missions that did not participate in the training, the average movement prediction error in the pre-training model decreased by 41 percent. After completing the pre-training, the team continued to use HiFi-UMI data for special post-training.
In this training system, pre-training and post-training play different roles. Pre-training helps the model to accumulate vision across tasks, experience of movements, general patterns of interaction such as learning to grab, move, align, place, and hand-to-hand collaboration; post-training allows the model to adapt to specific objects, task steps, positioning and termination conditions through high-precision specialized demonstrations.
As the amount of data increased, the model decreased the overall movement prediction error on the retention set by 61% and showed a clear downward trend, with α at 0.268 and R2 at 0.993.
This is not enough to demonstrate that the Scaling Law, which is widely used in the field of physical intelligence, has been developed, but at least it shows that within the current 4000-hour data range, the error of predicting the movements of the model can decrease steadily as high-quality UMI data increases.
The single-task experiment further demonstrates the improvement in data utilization efficiency from pre-training.
As shown on the left-hand side of the diagram, the model success rate increased from 37.5% to 65% after 800 items, and from 1600 to 70%, respectively, and from 3200 to 85% when 400 items were used for training tasks after remote control insertion. The success rate did not continue to increase when the data volume continued to increase to 6400 items. This shows that the post-training data gains in a single task will gradually decrease after the data reaches a certain level.
Not only that, but the performance of the pre-training model in ten tasks that did not participate in the pre-training also showed a clear substrate: the rigidity of the grappling task of the dining table type improved the fastest, the particles moved, and the clothing folding improved the slowest. The composition of the control pre-training data is not difficult to explain.
This reveals an interesting finding: the model's ability to migrate does not depend on "whether you've seen this object" but on "whether pre-training covers this type of physical interaction". Thus, the construction of a physical dataset cannot only statistically cover the number of tasks and the categories of objects, the underlying interaction patterns, the extent of state changes and failure branches, but it also determines what the model can learn.
Overall, the value of HiFi-UMI covers two key stages of robotic learning: the accumulation of replicable visual and action experience during the pre-training phase; and the rapid mastery and direct deployment of the model during the post-training phase. If post-training proves HiFi-UMI "available", then large-scale pre-training proves its "scalability".
Data routes or will determine the speed of expansion of physical intelligence
According to official sources, the current HiFi-UMI data production system has processed more than 20,000 hours of operating data, covering more than 480 scenarios.
The launch of the open-source HiFi-UMI-2K, a 2000-hour subset of high-quality screening and distribution balanced, includes six-track synchronized video, labeled hand tracks, claw states, language descriptions, and submission boundaries.
Open source data sets:https://huggingface.co/datasets/simple-world-lab/HiFi-UMI-2K
For the industry, HiFi-UMI-2K is not just about adding an operating video dataset, it's about providing the industry with a set of data that can be used to validate unobtrusive training, visual motion modeling, multi-point imaging, hand-track forecasting, cross-scenario migration, and motion redirection.
More importantly, it has transformed "highly reliable, unbiased data that can be accessed directly from a machine" from a company's internal experiments into a question that can be replicated and tested by more teams.
Machine Heart noted that the work quickly gained the attention and attention of the industry after its release.
As of this writing, HiFi-UMI has topped the Hugging Face Daily Papers daily list, with the open-source HiFi-UMI-2K data being downloaded 340,000 times in a week.
Additionally, AI content creator AK shared HiFi-UMI on X, and related posts were viewed about 50,000 times.
The continuing spread of heat, which no doubt illustrates this work, strikes a common problem with physical intelligence: while the industry needs larger real-world operational data, it cannot sacrifice the trajectory, timing and accuracy of interaction required for real-world execution.
In the past, these two goals have often been pursued by different data routes. HiFi-UMI attempts to integrate scale and reliability into the same set of data production systems. It also demonstrates to the industry that the Scaling Law of physical intelligence depends not only on model parameters and the number of robots, but also on whether a low-cost, high-reliability, sustainable data production chain can be built.
In fact, the start of HiFi-UMI can be traced back to a physically intelligent game at the end of 2025.
At the time, the Shenzhen Intelligent Team participated in the first BEHAVIOR Challenge jointly launched by the Lifeifei team and Stanford AI Laboratory. The difficulty of this competition was not uncommon in the field of physical intelligence: the entrance model required 50 complete home tasks in the BEHAVIOR-1K environment, from cleaning rooms to clean kitchens, each involving long-distance movement, arm operation and complex environmental interaction, a task that often consisted of a succession of dozens of links, where any step failed and the entire task was abandoned.
In the face of such a set of tasks, a common practice is to make specific adaptations to specific tasks and scenarios and to "modulate" the scores. But such high scores are often established only on the list, and change a task, change a scenario.
As the team recalled, more importantly, when the framework no longer relies on the characteristics of a single task, the ceiling will appear wherever it falls. The quality and scale of the data will become both specific: on the one hand, long-range tasks will gradually increase the space and timing of the sample data errors. On the other hand, small deviations in single-step movements are still tolerable, and once the task chain is extended, each step deviation will be the starting point for the next step, which is often interrupted in the middle; on the other hand, on the whole family of 50000 complete demonstration tasks, each task has an average of only about 200 items, while the cost structure of remote operation determines how difficult it is to overcome several levels.
At the time, it seemed that there was little way to meet both of them at the same time. No-body collection was the solution at scale, but its accuracy did not reach the requirements of long-range precision operations. Instead of continuing to press the costs on the remote side of the operation, it was better to raise the accuracy of no-body data.
In this context, it is easier to understand the position of HiFi-UMI in the overall technological route of Intelligent Thinking.
Shen Park Intelligent was founded in late 2024 to build a full-fledged loop around "models, data, bodies, scenarios". On the one hand, pre- and post-training of the body model, as well as Omni-Simple-World Model and Robotic Agent OS for long-term mission planning, cross-body adaptation and non-standard scenarios; on the other hand, the ongoing verification of real robots in the hotel, wellness and family business scenarios.
HiFi-UMI corresponds to the data base in this set of technology systems. The collection of real-world operations is extended by non-native devices, the data engine completes reconstruction, validation and marking, the model accumulates general experience through pre-training and mastery of specific tasks through post-training.
It is understood that Shen Park Intel is currently conducting POC testing on the hotel scene in an attempt to keep the new problems generated by real-machine deployments flowing back to data and model systems.
Finally, what I would like to say is that the real limitation of non-physical data to real deployment may not be the absence of a robot when it is collected, but the space, time and interaction information in the data is not "HiFi".
As this gap is gradually being filled, the competition for physically intelligent data will be rewritten. The next stage is probably not just who owns more robots, but who can produce data that is accurate enough, diverse enough, and can actually be transformed into robot-enabled operations at a lower cost, more efficiently.
When this data production chain begins to scale, the process of moving physically intelligent to real-world deployments on a massive scale may be just beginning...