Taking a newborn egg requires the right amount of force: both to prevent slippage and to prevent it from shattering. A wet towel is worn, and the force path of the fingers is constantly adjusted as the water is squeezed out. For humans, these operations are almost instinctive. But if the camera only relies on vision, it takes the trajectory of the lower arm, but cannot capture the position of the finger contact and the pressure.
In the past few years, in academia, first-person human video has expanded rapidly to tens of thousands of hours, and more and more work has begun to see the benefits of data growth. The size of visual and touch data is far from the same: previously publicly available human visual data sets are usually only a few hours to tens of hours.
Recently, researchers from 15 agencies including Texas A&M University of Agriculture, Google DeepMind, Carnegie Mellon University, Stanford University, UC Berkeley, Yale University, Microsoft, NVIDIA, Liverpool University, Meta, Washington University, Northwestern University, Sony, Georgia Tech, and Overfit Lab (the data provider) released TouchScale: a collection of 500 hours of human vision - sensing and touch data built on a unified sensor, unified collection and synchronized process.
The team hopes to answer a key question: If we no longer spell out data from different sources, but instead use the same set of sensing and retrieval systems to really extend high-quality first-person view to hundreds of hours of touch-to-touch data, will it, like large-scale video, show the benefits of scaling as the data grows?
What does TouchScale bring?
The TouchScale contains 500 hours of human hand-held visual-touch synchronization data, all collected from the same set of wearable systems, and synchronizes the head RGB-D, wrist RGB and hand-held touch signals.
2. Human "hands" can migrate to robots. No human action labels are required, and no human handcraft is mapped to robots, with only human visual and tactile data in mid-training, and the average success rate of the robot's 4 real-machine contact missions has risen from 22.5% to 57.5%.
3. Hundreds of hours of human visual-touch data began to show significant Scaling gains. TouchScale was used in mid-training, with real-machine operations becoming stronger as the data scale expanded overall; the same upward trend was observed in independent zero-sample touch prediction experiments.
Subject: TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning
Link to the paper:https://arxiv.org/abs/2610.10288
The main page of the project:https://touch-scale.github.io/
Address of data collection:https://huggingface.co/datasets/2077AIDataFoundation/TouchScale
500 hours of tactile data is more than just "taking more".
For TouchScale, the real challenge is not to simply stack the number of hours to 500, but to continue to expand under the same set of sensors, the same set of synchronous and collection processes. Previously open human visual-touch datasets were generally less than 30 hours; if several small datasets were directly packed together, different hardware and collection processes would change with the size of the data, making it difficult to study the impact of the scale itself separately.
TouchScale thus fixed the hardware from the receiving end. The head RGB-D camera records the overall operation process and the depth of the scene, while the wrist RGB camera on both sides closely records the interaction between hands and objects; both hands wear full-handed touch-sensing gloves, each containing 880 touch-sensing units (taxels), covering the fingers and palms, for recording contact pressure.
To ensure the quality of large-scale touch data, TouchScale combines hardware and algorithms to control the quality of the data. The collection ends use high-precision, low-noise flexible skin touch sensors, which steadily record pressure changes in the process of interaction between fingers and palms; after the collection is completed, the automated mass inspection process assisted by VLM, which only determines the grip, release and participation of the fingers in contact with the wrist video, and should be crossed with the actual touch.
In 500 hours, what did you get?
TouchScale contains about 87,000 interactive recordings, about 2,000 mission descriptions and more than 1,500 objects, covering 9 types of collection environments, more than 800 scenario configurations, and about 20 collectors participating in the completion of the task. It includes both daily operations and more detailed contact with tool use. For example, pouring water from a water tank into a wide-mouth bottle requires continuous adjustment of the container posture; rolling up laptop charging wires and attaching them magically, involving hand-to-hand collaboration and continuous contact; sealing the top of the container, which includes tools grab, positioning, and continuous pressing operations. Different tasks, objects, and execution methods also bring different - object postures and touch-touch modes.
Compared to open human visual-tactile data, TouchScale not only extends the scale to 500 hours, but also maintains a consistent sensor and collection process and offers hand-to-hand tactile collection, with 880 taxels per hand. This provides the conditions for studying the impact of the data scale itself separately.
What is the result of 500 hours of data from perception to reality?
16 hours of trans-sensor amplification is also better.
The first group of experiments examined zero samples of cross-sensor touch predictions. The model predicted hand touch from first perspective and wrist video, and tested directly on the EgoTactile, which was collected from another set of touch gloves, without the use of target data mitigation.
First, the amount of training data is controlled in the same way for about 16 hours: using a TouchScale subset of training models, contact cIoU (contact intersection over union) reaches 0.181, higher than EgoTouch's 0.134.
Touch control can also help with vision.
The second group of experiments used touch as a surveillance signal for the visual encoder, and then transferred the learned visual characteristics to the motion-recognition task. The researchers used TouchScale, OpenTouch, FEEL and EgoTouch respectively for pre-training and maintained the same initialization and training budget.
On the three benchmarks of MECCANO, Something-Something V2 and Ego-Exo4D, online detection and end-to-end fine-tuning of visual encoders with TouchScale pre-training achieved the highest accuracy compared to pre-training using OpenTouch, FEEL and EgoTouch.
Finally, let's look at reality: Can human "hand senses" be used by robots?
Using the xArm6 mechanical arm and BrainCo Revo 2, the real-world experiment set up four contact-intensive tasks: soft-hard object sorting, bottle cap removal, test tube transfer, and whiteboard wiping. Each task used 50 robot samples and performed 20 tests.
The control setting is simple: both versions start from the same N0-VTLA checkpoint, use visual and tactile inputs, and post-train with exactly the same robot demonstration. The only difference is that one version goes through TouchScale mid-training before post-training.
TouchScale mid-training does not use human motion labels, nor does it map human hand trajectories into robot movements. It uses N0-VTLA's future touch prediction targets to allow models to predict subsequent touch changes based on current vision, language, and touch. In other words, robots learn from human data not how the hand moves, but how the mouse touches.
After joining TouchScale mid-training, the average success rate for all four missions increased from 22.5% to 57.5%.
Soft-hard objects sorted: 10% → 60%
The bottle cap removed: 40% → 70%
Test tube transfer: 30% → 60%
The whiteboard is wiped: 10% → 40%
 |  |
The more data, the more the effects will continue to fade.
What's really exciting is that as the size of TouchScale's data continues to increase, models and robots continue to improve.
In touch prediction tasks, the model and rating settings remain unchanged, only adding training data. After the TouchScale was expanded from 10% to 100%, the cIoU rose from 0.311 to 0.383. That is, the more human visual-touch interactions we have seen, the more accurate contact predictions we can make when faced with a set of touch sensors we have never seen before.
The trend has not stopped at the perceptual level. The real-world demonstration data used by robots has remained unchanged, adding only TouchScale mid-training human data, and the average success rate for four missions has risen from 22.5% to 57.5%.
In the end, he wrote: "The touch also goes to the beginning of Scaling".
TouchScale wants to answer the question of what can be achieved by simply expanding the size of human visual-tactile data if the sensor and the collection process are consistent. The answer now seems pretty clear: as the amount of data increases, the overall accuracy of cross-sensor touch predictions becomes more accurate, and robots with TouchScale mid-training gain more real-machine capabilities.
This means that touch can no longer be just an expensive, scattered add-on to a robotic system. It can also be scaled up like first-person video, studied on a scale of chemistry, and further transformed into the physical interaction experience that robots need. More importantly, TouchScale shows not just a larger data set, but a new path to data expansion: when the sensors in human operations are systematically recorded, robots can get a portable physical interaction experience from these human experiences without action tags.
Of course, this path has only just begun. Current real-world experiments cover only one robotic platform and four contact-intensive tasks, and there are many questions to be explored about how the human touch experience translates into better robotic controls. But TouchScale has already given a clear signal: the path of Scaling that vision has already taken, touch is starting to take hold.
Introduction by the author
The first co-authors of the paper are Dayou Li, PhD student at the Texas Agricultural Institute of Technology, Hao Wang, a senior research engineer at Google DeepMind, Qianqian Yang, a Ph.D. student at Carnegie Mellon University, and Zihao Zhu, a Ph.D. student at the Texas Agricultural Institute of Technology.