“ It's a koala sleeping bag that allows robots to feel touch and emotions throughout their bodies.
The problem was solved by Mapp, who used a fully fledged coola sleeping bag to "paste skin" the robot.
The touch status of most social robots is: the widely used NAO has only a few switched touchpoints on the head, hands and feet, it's not easy to tell what movement you're doing or how hard you're using it; the PARO sensors are only installed on the neck and head, and there's no "blind spot" for both humans and animals.
The MAP Intelligent Systems Institute (University of Tennessee-Fraunhofer IPA) has developed a prototype called HERA: 16 self-made soft touch sensors are attached to the NAO, a modified Kola sleeping bag is worn to allow it to recognize five touch gestures and two strengths, and a "mass-bounce-inhibition" model allows emotions to continuously fluctuate over time.
The research has been published in IEEE Transactions on Affective Computing, entitled: Creating an Affective Robot That Feels Both Touch and Emotion.
01.
16 pieces of cloth, creating a layer of skin that can be touched
The principle of the sensor is not complicated: two layers of high-conductive fabric with an insulating coating, one layer of low-conductive aluminum foam cotton in the middle, the two layers approach when pressurized, the resistance decreases, and the voltage can be measured.
It is difficult to attach the curved surface of the body. The team's practice is: first press the mold on each part of the body, disassemble and scan it into reproducible versions in CAD, then automatically generate sizes according to the thickness and curvature of each layer, and finally cut the batch by laser cutting.
The 16 sensors cover the face, forearm and left sides of the shoulders, upper arms, lower arms, hands, thighs, lower legs, and feet. The joints are not intentionally covered to avoid scarring, and the back is not fitted because the user does not want to touch the microcontroller there.
Image: NAO with 16 flexible touch sensors
The previous generation's problem was that the clothes were too tight, the shoulder sensors were prepressed, the team removed another of the same supplements to relieve shoulder tension, added a back strap, and transformed the original head of the hat into a style that could fully pack the head. Interestingly, the garment also expanded the range of perception so that it combined the exposed area along the chest with the nearby sensor mechanics.
Chart: HERA on a customized Koala sleeping bag
By contrast, the iCub's touch skin has more than 2,000 touch units, costing about 250,000 euros; HERA is another touch screen that sacrifices position resolution inside a single unit of the sensor for low cost, ease of manufacture, and high performance. This is based on a previous interview with autism therapists: their priority is simple, durable, and inexpensive.
02.
First decide if it is touched, and then decide how it is touched.
The training data was collected by three experimenters, each touching for five seconds, gestures for striking, tapping, touching, squeezing, squeezing, strength, lightness and usefulness, positioning with 16 sensors plus a "nowhere to touch", a total of 1,020 times, available 933 times. The test data was much more difficult: the two experimenters touched different parts at the same time according to their instructions, the time set to be 4/6/8 seconds and misplaced, covering four situations "no one touches, only A, only B, two people" at the same time.
In the model, the team replicated the classic path of the "Statistical Feature + Random Forest" and found a serious adaptation, transforming it into two phases of convolutional neural networks: touch detection class two, followed by 10 class gesture recognition of the signal determined to be touch. This both mitigated the imbalance of about 1:16 in the negative positive sample, and allowed each channel to make independent judgments, thus supporting multiple body points at the same time.
Results: Touch detection of F1 is 0.773 and the real-time test set accuracy of gesture recognition of 52.2% is 10%, i.e. 5.2 times that of random, equivalent to work already done in this field.
It is also worth mentioning the three evaluation perspectives proposed by the team. Time neighbourhood: people do not strictly stick to the suggestion of stopping, bubble bullets also take time, so when judging refer simultaneously to the current and previous time window. Space neighbourhood: sensors activated in real-time tests are more than expected, screening confirms that not electrical interference but mechanical clutter.
Re-evaluation after substitution: Touch detection of F1 improved to 0.896, and gesture recognition accuracy increased from 52.2% to 86.6%. Most of the errors were due to stall time lag, mechanical coupling and a confusion of strength with similar gestures, rather than being recognized as completely unrelated touches.
03.
A set of emotions given to robots by the use of spraying and blocking
The team believes that the tactile response of existing robots is largely a one-time, event-like confirmation, with a lack of time dynamics. The scheme comes from the approach-avoidance theory of psychology: positive to stimulus triggers approximation, negative to stimulus avoidance, and strong emotions are often double-sided with negative stimuli removed, and emotions not only return to neutrality but also briefly bounce back to positive, and vice versa.
Zero-stage systems have no memory, no reaction moment; one-stage systems have memory but no oscillation; only two-stage systems can satisfy simultaneously. Hence, HERA's mood is modeled as a mass-bounce-inhibition system: the position of mass blocks represents the current effective price, with each touch transformed into a positive or negative pulse, and multiple simultaneous pulses, combined with simultaneous negative touches, can offset each other.
The three parameters correspond to three "characteristics": the natural length of the spring determines the default mood that returns when no one is disturbed (sex), the natural frequency determines the amplitude and speed of the response (slowness), and the impedance to determine whether the oscillation is balanced (stability).
The team performed a six-hour presentation at the institute's public event, with the NAO sitting in a fixed position, next to the screen showing the classification results and the white faces that would change faces in real time, set to grit and beat negative, squeeze touching grit, and the overall event was attended by about 3,500 people. The software was almost full at 40Hz, but more than a dozen sensors would be slightly delayed when touched simultaneously; the classifier would be briefly misreported at the end of some touch, but these times the neighborhood was flattened by the mood model, and the audience did not notice.
The team also acknowledged the limitations: the three low-sensitivity sensors, such as the legs, had to be redesigned due to the oxidation of conductive fabrics that led to an increase in baseline resistance, a decrease in sensitivity; this interaction was always a single gesture, and once the robot moved, the false touch of self-contact would be a new challenge; the strength was currently only two degrees. This demonstration did not constitute a formal user study, and HERA's potential as a treatment tool for children with autism still needs to be rigorously tested experimentally.