At the World Robotics Congress, touch perception has become the most-watched technology in the world. More and more practitioners are realizing that learning based solely on visual simulation is difficult to support robot landing, and touch as the core sensory of physical interaction is the key to breaking the bottleneck.
Exhibition exhibits are full, but the exit of the exhibition hall back to the real production line, landing points are still noticeable: a large number of robot business cards in the bulk delivery link, laboratory algorithms are difficult to adapt to complex industrial environments.
The instantaneous force control, anti-wrinkle, and material recognition problems of robots in contact with objects are currently high risk areas for missions to fail. Vision is unable to capture contact feedback, simulation learning is difficult to replicate human tactile experience, and robots are very likely to err in the face of unknown objects and complex working conditions.
Building on this industry's pain point, he is deeply engaged in touch track technology, relying on mass-produced touch sensor products, and at the WRC he has demonstrated a self-learning touch-to-touch shut-loop program at the WRC that attempts to address the underlying problem of the lack of physical interaction capability of robots.
01.
It is not enough to understand the physical world by visual imitation alone.
While the mainstream path to physical intelligence is to mimic learning ants, which collect human operational data, allowing robots to mimic it. The path has been verified in a laboratory environment, but still faces some obvious limitations when it comes to real-world scenarios:
The first is that visual data can't fully carry physical interaction information. When humans pick up an egg, they can sense the required force, adjust their grip gestures in real time, and this kind of tactile feedback can't rely on visual capture.
Second, the process of human transformation involves loss of information. Human presentation data needs to be converted into a format that can be recognized by the robot, but the flexibility of the human finger, the freedom of the joints, the layout of the sensor and the robot itself are naturally different, and the transformation link is easy to lose a lot of interaction details.
Furthermore, the ability to generalize models is short-lived, and there is insufficient adaptation to the scene. Once the crawling model has been trained in the laboratory, it is easy to see a significant decrease in the success rate of the task when the shape, material and posture of the object changes once it enters the actual production line.
The root of the syndrome is that robots lack the cognitive ability to adapt to their own bodies to understand the physical world, and lack the native interaction of data to support long-term continuous learning.
02.
How can a robot have "its own senses"?
The first step is to get the robot out of the dependency on human demonstration data, self-understand the physical world, build a tactile sensing system that adapts to the body itself and is able to capture information about mechanical interactions. The robot wants to get the original biological interaction data on the condition that it has a set of its own tactile sensors. At this WRC, he demonstrated a full range of tactile sensing products, building a set of tactile sensing matrix that covers the entire body of the robot.
The complete set of products follows the idea of a layout that extends from local fine sensing to whole-body safety sensing. The first landing, and the most central scenario of operation interaction, is the mechanical hand touch solution. The TS-F series of full-hand sensors breaks the industry's traditional approach to long-term fingerprint single-point sensing, achieving fingerprint, thumb-to-hand area touch coverage, restoring human hand tissue touch distribution logic; has a 0.01N high-precision resolution, can identify more than 30 different materials, and is equipped with 1 cm of proximity to sensing, and has been tested for a long time after 3 million movements, 1000 N uploaded.
Among these, the TS-F fingerprint touch sensor further integrates proximity sensing, three-dimensional force detection, temperature difference and texture perception, can predict the distance of the object before contact with the object, allowing the robot to adjust the gesture in advance, plan the operation path. On this basis, the transitional TS-FT multi-modal sensor, new temperature sensing dimensions, can capture temperature changes between 40°C and 120°C in 3 seconds, achieving touch and temperature sensing fusion. This is not a mere technological superimposition, but a multi-dimensional sensing of human markers, broadening the sensing boundaries of the robot's physical environment, and continuously improving the human touch system.
When a mechanic completes refined operational perception, the robot's body also needs a comprehensive safety sensing capability. TS-EST multi-mode electronic skin supports the robot's full-body deployment, while maintaining a power resolution of 0.01N, a temperature detection accuracy of 0.1°C, and an approximation sensitivity of more than 1 cm, allowing the robot to make safety predictions before a biological collision, while maintaining operational flexibility and operational safety.
Sensing hardware continues to produce high-quality touch signals, with the computational support of the underlying chips. The Greenstone E10A chip is the world's first dynamic touch sensing chip, using ASC technology to directly convert touch signals into pulse signals, adopting event-driven, sparse computing architectures, achieving micro-second responses with ultra-low power consumption. Like the human nervous system, robots do not have to wait for the central processor to unify operations to react quickly to touch stimuli, significantly shortening interaction delays.
Based on a complete technical matrix, the products of He Mountain Technologies have been widely verified in the industry chain, and have now reached deep commercial collaboration with more than 180 customers in the robotics industry chain at home and abroad, covering Galaxy General, Automation, Stellar Age, and other robotics manufacturing enterprises, as well as manufacturers of robots, high-technology, smart smart, and agile handsets, to prove the geographical value of touch sensing software in the industry.
03.
How can physical interactions be transformed into traceable data?
A new consensus is emerging in the field of artificial intelligence: simulation learning, which relies solely on human video and action demonstrations, is nearing its peak to push robots into the real world, to scale up the collection of native multi-modal physical interactions of the robot's body, and to become a competitive focus for the next stage. However, the reality is that current touch data collection is still the industry's flagship shortcut: traditional data collection methods largely depend on human demonstrations and information loss; touch data lacks standardized collection, labeling tool chains, and data is difficult to replicate; visual and touch modeling disintegration is not possible to synchronize with real interaction recording processes.
With precision touch sensing hardware, the next step is to open a link to the conversion of physical interaction information to training data. To address the industry data pain points, he launched two complementary programs, respectively, aimed at human-machine synchronized data collection and robot autonomous loop data collection.
The TS-ECHO touch-sensing bracelet provides a lightweight solution for the human-machine synchronous pickup scenario. The device is worn on the operator's fingers, synchronously records interaction states such as contact, force, slippage during man-made operations, produces trainable, synchronous, markupable high-quality touch data, and supports the integration of visual, motion, and language modules. The system is able to replenish missing magnetic information in pure visual demonstration, serving robotic operating learning, world models, VTLA, and other advanced research directions, and pushing robots to understand the laws of physical interaction from mere sight to worldview.
The TS-V focuses on landing applications, integrating visual and tactile multi-modal sensing, enabling robots to achieve stable grappling of all kinds of objects in complex environments, real-time sensing of contact states and dynamic changes, and adaptation of claw movements; while the TS-VT focuses on the underlying data base, providing specialized multi-dimensional touch data recording, marking and data extraction capabilities, compatibility with multi-modal mainstream handhelds, claw end executors, stable capture of three-dimensional sensing, proximity sensing, touch point variation, and other key parameters, providing a standard data base for sensing touch computing training and computer performance evaluation.
But with a complete physical interaction data conversion system, it is difficult for a single enterprise to complete independently. Touch data labeling, cleaning the door is much higher than visual data, the industry has not yet developed a mature tool chain, wanting to achieve synchronous collection of multi-modal information with touch, integrate duplication, and need to complement the multi-faceted capabilities of the industrial chain.
Based on a well-established deployment of AI in the field of machine vision and in-vehicle optics, both sides have opened up data links for visual imaging and touch sensing, jointly developing multi-modal perception schemes for industrial and service scenarios adapted to complex operating environments, enabling visual trajectories and tactile information to be captured and synchronized.
He co-created data for industry-wide data labeling, data set management shortcuts, he teamed up with Mountain Tech to create data, integrate one another's Ego perspective eyelash collection equipment with his Mountain Touch fingerprint, create a joint data collection program, build a full-link tool chain covering raw data collection, multi-module integration, data labeling, data set management, and effectively reduce barriers to industry use of multi-module interaction data.
At the same time, the jointly-owned robotic vision enterprise Obi-Light, which plays the natural complementary advantages of visual distance perception of the contour position, touch proximity to determine the hardness temperature dynamic state, achieves deep integration at the sensor layer, data layer, and algorithm level, and breaks down the multi-modal data cracking problem from the bottom.
He synchronized new products such as Mountain Science Touch Sensing Fingerprints on the Jiktong platform, bringing professional touch sensing products from the B-end industry market to a broader integrated channel, showcasing touch data collection locally to consumer-level developers and small and medium-sized enterprises, and expanding the reach of the touch technology market.
From optical vision, data marking tools to multi-mode perceptual integration, a range of ecological collaboration and continuous complementation of touch technology landing support capabilities. External collaboration with his self-developed touch hardware, data collection programs, and reinforcement training underpinnings, together promoting a complete, physically intelligent touch ecosystem, continuously reducing barriers to industry access and use of raw biological data interaction.
04.
Can robots evolve from "imitation" to "self-learning"?
With perceptual hardware and continuous interaction data, the core thesis follows: how to make robots truly master their operational capabilities. This is also the starting point for the team of Richard S. Sutton, the father of AI reinforcement learning, who co-founded the AI robotics kindergarten.
The underlying logic of simulation learning is to replicate human experience, whereas reinforcement learning relies on the autonomous accumulation of experience by robots. The biggest advantage of reinforcement learning is that it has greater generalization capabilities, and that robots learn general operating strategies through continuous error trials, rather than solidifying action templates, to be more adaptable to new scenarios. However, this path has a significant shortcut in the long run: mass testing, iteration, implementation of high-cost training cycles, and large-scale testing directly in real-world physical environments are not easy.
This is precisely the paradox that is targeted at robotic kindergarten: building a feasible program that enables reinforcement learning to be implemented in real physical environments.
Based on his self-development in touch-sensing technology, he created four core conditions for robots: high-performance error-testing environments, step-by-step learning pathways, sustained multi-modal real-world interaction, and in-motion-driven adaptive feedback. Combined with Sutton's reinforced learning theory, robots can, like human children in general, rely on continuous interactive feedback, progressive trial and error development in real-world physical environments, and gradual learning of general operational abilities and environmental adaptability.
On the WRC site, a robotic intuition with a touch-sensing system demonstrates this evolutionary idea, which can be clearly divided into two phases. The first phase is basic intrinsic safety perception: robots have the ability to touch feedback similar to that of a touch-sensing pain reader, contact with hazardous objects or adjust their movements in a timely manner when exposed to external forces, and prioritize assuring intrinsic safety, which is a prerequisite for continuous autonomous learning.
In the field of interactive demonstration using the new electronic skin TS-EST, robots combine simulation learning and reinforcement learning in the course of walking, relying on tactile signals to produce step-by-step feedback in real-time. Unlike the show routes that rely on tactile vacuum flipping to attract attention, the program demonstrates progressive, continuous evolutionary paths. But objectively, reinforcement learning still requires a lot of trial and error, and robots will still experience imbalances and falls during training. The key difference is: relying on continuous feedback, robots can draw effective information from each interaction, continuously modify strategies, and gradually improve environmental adaptation.
The Botanical Gardens will officially open on September 1 and will continue to attract eco-partners from the industry.
05.
I'm going to tell you about the future.
With physical intelligence, there is no shortcut to the real-world scenario. The industry is now generally concerned about when robots will be able to do the real work, while hill science tries to solve the more fundamental questions: how robots continue to learn, master the ability to interact physically. Robots want to truly understand the physical world, the key may not be to create more sophisticated mechanical structures, but to build a technological system that allows robots to accumulate experience independently and continue to generate.