Scientists put a "child" in a baby's bed.
An elastic cord connects the mattress to one of its wrists, and the mattress follows the mattress.
It's not a baby in bed, it's an iCub doll robot that looks like a child.
The study, by a joint team from the University of Hamburg, the Czech Technical University in Prague and the University of West Germany/CNRS in Paris, was published yesterday in the leading international journal Science Robotics.
They're trying to figure out an old question in developmental psychology: how do babies find out, "I move, the world changes?" and this time the way to answer it is to get robots to lie in their beds and really play.
01.
It's a classic experiment in developmental psychology, and it's going to be done by robots.
The mobile paradigm, proposed in 1969 by Rovee and Rovee, is one of the most classic methods of studying the association of baby sensory motion with sensorimotor contingency (SMC).
The procedure is simple: the baby lies in a small bed with only one limb attached to the top of the bed with a rope. By 3 to 4 months of age, the baby finds himself making the bed move, then moves more of the "attached" limb. If the rope is untied at this time, some babies will experience a "retreat burst".
But this phenomenon is not always observed, and there is a huge difference between babies.
Several previous computational models have attempted to explain this paradigm, including the reinforcement learning class, the dynamical system class, and the physical model. But they have all been run in a simulation, with no real-world noise, latency, and physical constraints, and no way to incorporate predictive errors and curiosity, the cognitive mechanisms that are thought to support human behavior.
The team moved their previous simulation model to the real machine. The iCub used three degrees of freedom on the shoulder, one elbow, one wrist, and five joints in total; vision only with the camera of the right eye, with a resolution of 320×240, 15 tons, and after being polarized into 70 units of vision, the model's input was "how much proportion of the graph changes in the image".
There are seven conditions in the experiment, three on the machine and four on the simulation.
The real part of the machine: the complex condition uses a multi-part mattress that is close to the original experiment, and the movement is quite chaotic after being pulled; the simple condition replaces a bubble tube suspended by a skid, the mode of movement is much more determined; the control condition simply does not hang the mattress. The simulated part includes the simulated complex; the simulated simple, which moves the visual input with the wrist position to 1:1, and the two "relay conditions the motion of the mattress is reproduced from previously recorded data, the image stimulates the same pattern, but is completely unrelated to the robot's actions".
Each condition is repeated multiple times: real machine complex and simple 20 times, control 10 times, and four simulation conditions 60 times.
On the model side, the robot's "brain" is a real-time trained neural network that enters 19 values (eighteen joint values from the last action plus past image changes), the middle two layers of 3000 and 1100 neurons, outputting 990+18+1 values, 990 motion instructions, 18 joint value predictions, and one image change prediction.
Population coding: Each neuron simultaneously affects three joints in the same limb, plus a random noise of ±0.05 to simulate the movement variability of babies. That is, robots have to learn for themselves what the relationship between their joints and limb movement and environmental effects is.
Learning from two monitoring signals: predicting losses; predicting both one's own movements and the environment's responses; and curious losses; exploring unpredictable outcomes; hence, "interesting" movements.
02.
Robots do learn, but not by "moving more".
How do you know if there is a "discovery" link between robots?
The team's idea was to train a multi-layered sensor (MLP) as an external observer. Show it the sequence of 10 consecutive positions of the two arms and let it decide whether the left or right hand was connected to the mattress at the time.
The results are as follows:
Real machine complex conditions are 65% higher than random; simple conditions 53%, control conditions 48%, no more than random. Simulated complex 67%; simulated simple 58%, all significant; while the two relay conditions fall to 45% and 48% respectively.
In the seven conditions, there are four related conditions that exceed the random level, none of which are unrelated.
Interestingly, an anti-intuitive law emerged: the simpler, the more definite the conditions, the more difficult it is to classify. The team's explanation was that complex conditions happen to be in a "difficulty sweet zone" that is challenging enough to trigger proactive exploration, but not so much that it will never be learned.
According to the classic assumption of infant literature, it should be the hand that connects more.
The team measured the length of the path through the wrist.
The result was: no noticeable difference. The total path length of the connected and non-connected arms is almost the same regardless of the conditions. The only consistent phenomenon is that, under the associated conditions, the actual machine takes about 200 steps and the simulation takes about 450 steps, and the volume of motion changes abruptly.
In other words, what MLP gets is not "too much movement".
03.
The two strategies are predictive and curious.
So the team switched to a decision tree, converted the trajectory into a series of motion indicators, which regions in space, variance, correlation between the two arms, and so on, and let the classifier find the boundary.
Under complex conditions, four dominant strategies covered 87% of the sample.
For example, if the left arm is active in a small area near the rib cage (nearly 9 cm), it is determined that the right arm is attached to the rib cage because in that position the rope is loose and the left arm will not move the mattress again. The other is a strategy that covers 75% of the left hand connection sample: the attached arm stays away from the rib cage, the area where the rope is tight, and only very small movements are needed to make the bed stiffen.
There are two other strategies that are a massive exploratory movement, not necessarily just the hand that connects.
The four strategies can be divided into two categories: either large-scale, large-scale exploration, or small, precise actions in specific areas of tension on the rope.
More importantly, the second decision tree team used the model's internal variables (two fractions of predicted losses, curious losses, image changes, targeted link values) to train a classifier to find the internal states of the eight groups and then calculate their correspondence to the conditional probabilities of external behavior strategies.
The first two "precision-small-motion" strategies are optimized at low losses: the model has learned to control the body, can accurately predict environmental responses, and is no longer surprised, so it switches to "maintaining the status quo" and holds the association in its hands with minimal movements.
The latter two "widespread exploration" strategies emerged at a time when models were not yet well-trained before the movement: action predicts excessive losses, behavior driven primarily by curiosity, and therefore substantial variation, not limited to disruption of the connecting limbs.
This is exactly the same as reinforcement learning: exploration vs. exploitation.
It is also consistent with infant data. Watanabe and Taga's 2006 study found that babies who are 2 months older and are not yet fully controlled for movement increase the movement of all limbs; babies who are 4 months older and have better mobility specifically increase the movement of the limbs attached to the mattress.
In addition, in some trials, robots have also experienced a baby-like, sometimes non-negative degeneration outbreak.
The team also conducted dilution and reduction degrees of freedom experiments, which were somewhat counterintuitive: the most difficult for the model was the motion control itself, while complex associations may have helped learn how to play because binary associations with only two states of "on/off" provide more operable possibilities, more precise proportional feedback, and richer stimuli to continue to trigger curiosity.
So the suggestion of this study to the infant researcher was: don't just use the "how much to move" indicator to judge whether the infant has learned. Experiments that exclude "not conforming to typical patterns" in the infant literature are another effective strategy. Researchers can in turn use the MLP and decision tree framework to more quantitatively sketch the infant's strategy, or even manipulate the three factors of "action predictability", "environmental predictability" and "curiosity", respectively, by adding weight to the body and adding noise to the mattress.
The paper's data and code have been made public.
Address of the paper: