Investigation of the Sense of Agency in Social Cognition, based on frameworks of Predictive Coding and Active Inference: A simulation study on multimodal imitative interaction

When agents interact socially with different intentions, conflicts are difficult to avoid. Although how agents can resolve such problems autonomously has not been determined, dynamic characteristics of agency may shed light on underlying mechanisms. The current study focused on the sense of agency (SoA), a specific aspect of agency referring to congruence between the agent's intention in acting and the outcome. Employing predictive coding and active inference as theoretical frameworks of perception and action generation, we hypothesize that regulation of complexity in the evidence lower bound of an agent's model should affect the strength of the agent's SoA and should have a critical impact on social interactions. We built a computational model of imitative interaction between a robot and a human via visuo-proprioceptive sensation with a variational Bayes recurrent neural network, and simulated the model in the form of pseudo-imitative interaction using recorded human body movement data. A key feature of the model is that each modality's complexity can be regulated differently with a hyperparameter assigned to each module. We first searched for an optimal setting that endows the model with appropriate coordination of multimodal sensation. This revealed that the vision module's complexity should be more tightly regulated than that of the proprioception module. Using the optimally trained model, we examined how changing the tightness of complexity regulation after training affects the strength of the SoA during interactions. The results showed that with looser regulation, an agent tends to act more egocentrically, without adapting to the other. In contrast, with tighter regulation, the agent tends to follow the other by adjusting its intention. We conclude that the tightness of complexity regulation crucially affects the strength of the SoA and the dynamics of interactions between agents.Comment: 23 pages, 8 figure

Turn-Taking Mechanisms in Imitative Interaction: Robotic Social Interaction Based on the Free Energy Principle

This study explains how the leader-follower relationship and turn-taking could develop in a dyadic imitative interaction by conducting robotic simulation experiments based on the free energy principle. Our prior study showed that introducing a parameter during the model training phase can determine leader and follower roles for subsequent imitative interactions. The parameter is defined as , the so-called meta-prior, and is a weighting factor used to regulate the complexity term versus the accuracy term when minimizing the free energy. This can be read as sensory attenuation, in which the robot’s prior beliefs about action are less sensitive to sensory evidence. The current extended study examines the possibility that the leader-follower relationship shifts depending on changes in during the interaction phase. We identified a phase space structure with three distinct types of behavioral coordination using comprehensive simulation experiments with sweeps of of both robots during the interaction. Ignoring behavior in which the robots follow their own intention was observed in the region in which both s were set to large values. One robot leading, followed by the other robot was observed when one was set larger and the other was set smaller. Spontaneous, random turn-taking between the leader and the follower was observed when both s were set at smaller or intermediate values. Finally, we examined a case of slowly oscillating in anti-phase between the two agents during the interaction. The simulation experiment resulted in turn-taking in which the leader-follower relationship switched during determined sequences, accompanied by periodic shifts of s. An analysis using transfer entropy found that the direction of information flow between the two agents also shifted along with turn-taking. Herein, we discuss qualitative differences between random/spontaneous turn-taking and agreed-upon sequential turn-taking by reviewing both synthetic and empirical studies.journal articl

Human-Robot Kinaesthetic Interaction Based on Free Energy Principle

The current study investigated possible human-robot kinaesthetic interaction using a variational recurrent neural network model, called PV-RNN, which is based on the free energy principle. Our prior robotic studies using PV-RNN showed that the nature of interactions between top-down expectation and bottom-up inference is strongly affected by a parameter, called the meta-prior, which regulates the complexity term in free energy.The study also compares the counter force generated when trained transitions are induced by a human experimenter and when untrained transitions are induced. Our experimental results indicated that (1) the human experimenter needs more/less force to induce trained transitions when $w$ is set with larger/smaller values, (2) the human experimenter needs more force to act on the robot when he attempts to induce untrained as opposed to trained movement pattern transitions. Our analysis of time development of essential variables and values in PV-RNN during bodily interaction clarified the mechanism by which gaps in actional intentions between the human experimenter and the robot can be manifested as reaction forces between them.Comment: 12 pages, 8 figures, journal pape

Emergence of sensory attenuation based upon the free-energy principle

The brain attenuates its responses to self-produced exteroceptions (e.g., we cannot tickle ourselves). Is this phenomenon, known as sensory attenuation, enabled innately, or acquired through learning? Here, our simulation study using a multimodal hierarchical recurrent neural network model, based on variational free-energy minimization, shows that a mechanism for sensory attenuation can develop through learning of two distinct types of sensorimotor experience, involving self-produced or externally produced exteroceptions. For each sensorimotor context, a particular free-energy state emerged through interaction between top-down prediction with precision and bottom-up sensory prediction error from each sensory area. The executive area in the network served as an information hub. Consequently, shifts between the two sensorimotor contexts triggered transitions from one free-energy state to another in the network via executive control, which caused shifts between attenuating and amplifying prediction-error-induced responses in the sensory areas. This study situates emergence of sensory attenuation (or self-other distinction) in development of distinct free-energy states in the dynamic hierarchical neural system

Incremental Learning of Goal-Directed Actions in a Dynamic Environment by a Robot Using Active Inference

This study investigated how a physical robot can adapt goal-directed actions in dynamically changing environments, in real-time, using an active inference-based approach with incremental learning from human tutoring examples. Using our active inference-based model, while good generalization can be achieved with appropriate parameters, when faced with sudden, large changes in the environment, a human may have to intervene to correct actions of the robot in order to reach the goal, as a caregiver might guide the hands of a child performing an unfamiliar task. In order for the robot to learn from the human tutor, we propose a new scheme to accomplish incremental learning from these proprioceptive–exteroceptive experiences combined with mental rehearsal of past experiences. Our experimental results demonstrate that using only a few tutoring examples, the robot using our model was able to significantly improve its performance on new tasks without catastrophic forgetting of previously learned tasks

Turn-Taking Mechanisms in Imitative Interaction: Robotic Social Interaction Based on the Free Energy Principle

This study explains how the leader-follower relationship and turn-taking could develop in a dyadic imitative interaction by conducting robotic simulation experiments based on the free energy principle. Our prior study showed that introducing a parameter during the model training phase can determine leader and follower roles for subsequent imitative interactions. The parameter is defined as w, the so-called meta-prior, and is a weighting factor used to regulate the complexity term versus the accuracy term when minimizing the free energy. This can be read as sensory attenuation, in which the robot’s prior beliefs about action are less sensitive to sensory evidence. The current extended study examines the possibility that the leader-follower relationship shifts depending on changes in w during the interaction phase. We identified a phase space structure with three distinct types of behavioral coordination using comprehensive simulation experiments with sweeps of w of both robots during the interaction. Ignoring behavior in which the robots follow their own intention was observed in the region in which both ws were set to large values. One robot leading, followed by the other robot was observed when one w was set larger and the other was set smaller. Spontaneous, random turn-taking between the leader and the follower was observed when both ws were set at smaller or intermediate values. Finally, we examined a case of slowly oscillating w in anti-phase between the two agents during the interaction. The simulation experiment resulted in turn-taking in which the leader-follower relationship switched during determined sequences, accompanied by periodic shifts of ws. An analysis using transfer entropy found that the direction of information flow between the two agents also shifted along with turn-taking. Herein, we discuss qualitative differences between random/spontaneous turn-taking and agreed-upon sequential turn-taking by reviewing both synthetic and empirical studies

Turn-Taking Mechanisms in Imitative Interaction: Robotic Social Interaction Based on the Free Energy Principle

This study explains how the leader-follower relationship and turn-taking could develop in a dyadic imitative interaction by conducting robotic simulation experiments based on the free energy principle. Our prior study showed that introducing a parameter during the model training phase can determine leader and follower roles for subsequent imitative interactions. The parameter is defined as w, the so-called meta-prior, and is a weighting factor used to regulate the complexity term versus the accuracy term when minimizing the free energy. This can be read as sensory attenuation, in which the robot’s prior beliefs about action are less sensitive to sensory evidence. The current extended study examines the possibility that the leader-follower relationship shifts depending on changes in w during the interaction phase. We identified a phase space structure with three distinct types of behavioral coordination using comprehensive simulation experiments with sweeps of w of both robots during the interaction. Ignoring behavior in which the robots follow their own intention was observed in the region in which both ws were set to large values. One robot leading, followed by the other robot was observed when one w was set larger and the other was set smaller. Spontaneous, random turn-taking between the leader and the follower was observed when both ws were set at smaller or intermediate values. Finally, we examined a case of slowly oscillating w in anti-phase between the two agents during the interaction. The simulation experiment resulted in turn-taking in which the leader-follower relationship switched during determined sequences, accompanied by periodic shifts of ws. An analysis using transfer entropy found that the direction of information flow between the two agents also shifted along with turn-taking. Herein, we discuss qualitative differences between random/spontaneous turn-taking and agreed-upon sequential turn-taking by reviewing both synthetic and empirical studies

Choice of biomaterials : Do soft occlusal splints influence jaw-muscle activity during sleep? A preliminary report

Aim: The choice of biomaterials for occlusal splints may significantly influence biological outcome. In dentistry, hard acrylic occlusal splints (OS) have been shown to have a temporary and inhibitory effect on jaw-muscle activity, such as tooth clenching and grinding during sleep, i.e., sleep bruxism (SB). Traditionally, this inhibitory effect has been explained by changes in the intraoral condition rather than the specific effects of changes in-occlusion. The aim of this preliminary study was to investigate the effect of another type of occlusal surface, such as a soft-material OS in addition to a hard-type OS in terms of changes in jaw-muscle activity during sleep. Materials and Methods: Seven healthy subjects (mean ± SD, six men and one woman: 28.9 ± 2.7 year old), participated in this study. A soft-material OS (ethylene vinyl acetate copolymer) was fabricated for each subject and the subjects used the OS for five continuous nights. The EMG activity during sleep was compared to baseline (no OS). Furthermore, the EMG activity during the use of a hard-type OS (Michigan-type OS, acrylic resin), and hard-type OS combined with contingent electrical stimulation (CES) was compared to baseline values. Each session was separated by at least two weeks (washout). Jaw-muscle activity during sleep was recorded with single-channel ambulatory devices (GrindCare, MedoTech, Herlev, Denmark) in all sessions for five nights. Results: Jaw-muscle activity during sleep was 46.6 ± 29.8 EMG events / hour at baseline and significantly decreased during the hard-type OS (17.4 ± 10.5, P = 0.007) and the hard-type OS + CES (10.8 ± 7.1, P = 0.002), but not soft-material OS (36.3 ± 24.5, P = 0.055). Interestingly, the soft-material OS (coefficient of variance = 98.6 ± 35.3%) was associated with greater night-to-night variations than baseline (39.0 ± 11.8%) and the hard-type OS + CES (53.3 ± 13.7%, P < 0.013). Conclusion: The present pilot study in small sample showed that a soft-material occlusal splint does not seem to inhibit jaw-muscle activity during sleep. Within the limitation of the study, it appears that the choice of biomaterials for occlusal splints may have a significant impact on the neurobiological regulation of jaw-muscle activity during sleep

Haemodynamic reactions in human masseter muscle during different types of contractions

Objectives : To investigate to what extent different types of jaw-muscle contractions cause haemodynamic reactions in human masseter muscle. Materials and Methods : Eleven healthy volunteers (seven males : 25.0±2.9 years and four females: 23.3±4.3 years) performed three standardized oral-motor tasks : maximal voluntary contractions (MVC ; duration 5 sec, 3 times repetition), tooth grinding (repetitive left and right side grinding from intercuspal position to canine-to-canine position at 0.5 Hz keeping 50% MVC for a total of 10 times), and 1-min left-side gum chewing at 1 Hz. Haemodynamic characteristics were measured in the left masseter muscle with the use of a laser blood oxygenation monitor (BOML1TRW, OMEGAWAVE INC., Tokyo, Japan). Electromyographic (EMG) activity from right and left masseter muscle was simultaneously monitored (500 Hz sample frequency) during the tasks. 1-ANOVA followed by Dunnett’s test was used. Results : Oxygenated haemoglobin (OXYHb : 13.5±0.2 104 units/mm3) and deoxygenated haemoglobin (deOXYHb : 7.6 ± 0.3 104 units/mm3) did not change significantly during the MVC task (13.9±0.2 and 7.8±0.3 104 units/mm3, respectively, P>0.065), however, the total haemoglobin (TOTALHb : 22.1±0.3 104 units/mm3) showed a significant increase (22.7±0.3 104 units/mm3, P=0.003) during the MVC. Tissue blood oxygen saturation was not changed during the MVC (P=0.164). During the tooth grinding task, OXYHb, deOXYHb, TOTALHb, and tissue blood oxygen saturation (StO2) remained constant (P>0.127). Finally, the chewing task was associated with significant decreases in StO2 (67.9±0.7%, P=0.006) related to a decrease in OXYHb (14.0±0.2 104 units/mm3, P=0.040) compared to baseline (68.8±0.7% and 14.2 ±0.3 104 units/mm3, respectively). Conclusion : These results showed that high-intensity experimental tooth clenching caused constriction-like reactions in the masseter muscle whereas tooth grinding did not cause detectable changes in haemodynamic characteristics of masseter muscle. Finally, the findings indicated that rhythmic dynamic contractions might lead to oxygen deficit in the masseter muscle. The present data may have implications for understanding the potential pathophysiological consequences of different types of oral-motor tasks, e.g., bruxism and prolonged mastication