Brain response to a humanoid robot in areas implicated in the perception of human emotional gestures

BACKGROUND: The humanoid robot WE4-RII was designed to express human emotions in order to improve human-robot interaction. We can read the emotions depicted in its gestures, yet might utilize different neural processes than those used for reading the emotions in human agents. METHODOLOGY: Here, fMRI was used to assess how brain areas activated by the perception of human basic emotions (facial expression of Anger, Joy, Disgust) and silent speech respond to a humanoid robot impersonating the same emotions, while participants were instructed to attend either to the emotion or to the motion depicted. PRINCIPAL FINDINGS: Increased responses to robot compared to human stimuli in the occipital and posterior temporal cortices suggest additional visual processing when perceiving a mechanical anthropomorphic agent. In contrast, activity in cortical areas endowed with mirror properties, like left Broca’s area for the perception of speech, and in the processing of emotions like the left anterior insula for the perception of disgust and the orbitofrontal cortex for the perception of anger, is reduced for robot stimuli, suggesting lesser resonance with the mechanical agent. Finally, instructions to explicitly attend to the emotion significantly increased response to robot, but not human facial expressions in the anterior part of the left inferior frontal gyrus, a neural marker of motor resonance. CONCLUSIONS: Motor resonance towards a humanoid robot, but not a human, display of facial emotion is increased when attention is directed towards judging emotions. SIGNIFICANCE: Artificial agents can be used to assess how factors like anthropomorphism affect neural response to the perception of human actions

Time course of upper beta rhythm for cyclic and no-cyclic movements.

<p>The graph shows the EEG power time course for cyclic (blue line) and non-cyclic (red line) observed movements. Asterisks indicate significant differences between the corresponding bins in the two curves. Each epoch is labeled by a different color.</p

Time Course of EEG Rhythms.

<p>The graph shows the EEG power time course for each frequency band: alpha band (8–13 Hz) in green, lower beta (13–18 Hz) in red, and upper beta (18–25 Hz) in cyan. Each epoch (fix, still, movement and post movement) is labeled by a different color. Significant differences between adjacent time bins are indicated by asterisks whose color-code corresponds to that used for different bands. At the top of the figure a film strip shows an example illustrating the different epochs of the observed video clips.</p

Significant Signal Changes in Subjects Receiving Explicit Instructions Compared to Subjects Receiving Implicit Instructions in the Three Tasks Versus Rest

<p>Threshold of <i>Z</i> = 2.3 at voxel level and a cluster level corrected for the whole brain at <i>p</i> < 0.05. The two black arrows indicate two foci of activity in dorsal premotor cortex that are located deep in the sulci and thus not easily visible on the three-dimensional surface rendering. See <a href="http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.0030079#st003" target="_blank">Tables S3–S5</a> for coordinates of local maxima.</p

Six Images Taken from the Context, Action, and Intention Clips

<p>The images are organized in three columns and two rows. Each column corresponds to one of the experimental conditions. From left to right: Context, Action, and Intention. In the Context condition there were two types of clips, a “before tea” context (upper row) and an “after tea” context (lower row). In the Action condition two types of grips were displayed an equal number of times, a whole-hand prehension (upper row) and a precision grip (lower row). In the Intention condition there were two types of contexts surrounding a grasping action. The “before tea” context suggested the intention of drinking (upper row), and the “after tea” context suggested the intention of cleaning (lower row). Whole-hand prehension (displayed in the upper row of the Intention column) and precision grip (displayed in the lower row of the Intention column) were presented an equal number of times in the “drinking” Intention clip and the “cleaning” Intention clip.</p

Presented stimuli and timeline of the experiment.

<p>In the figure the structure of the administered trial is shown. The upper part shows still frames from the video clips displaying the four different observed movements. Target and cyclic attributes of movements are showed in the rightmost part of the upper panel. In the first line of the lower part, the four epochs of each trial are indicated. The second and third lines show the time bins considered in the analysis and the corresponding time interval (each 200 ms long), respectively.</p

Areas of Increased Signal for the Three Experimental Conditions

<p>Threshold of <i>Z</i> = 2.3 at voxel level and a cluster level corrected for the whole brain at <i>p</i> < 0.05.</p

Velocity profile and beta rhythm time course for each single type of movement.

<p>The figure includes four panels (A, B, C, and D), one for each observed movements, showing the velocity profiles of observed hand (red line) superimposed on the EEG beta power (blue line). The velocity profiles were computed as the displacement of the fingertip of the actor between subsequent video frames. To make their values comparable, velocity data were normalized to their maximal value.</p

Time course of alpha-band power for target and non-target movements.

<p>The graph shows the alpha-band EEG power time course for target (blue line) and non-target (red line) observed movements. Asterisks indicate significant differences between the corresponding bins in the two curves. Each epoch is labeled by a different color.</p

128-channel EEG array.

<p>Top view of the scalp where the blue circles indicate electrodes located in the frontal clusters, red ones in parietal clusters. Green area includes left hemisphere electrodes, purple area the right hemisphere ones. The clusters includes C3, C4, P3, P4. In the figure electrodes belonging to occipital clusters (including O1 and O2) are not visible.</p