Structure Learning in Motor Control:A Deep Reinforcement Learning Model

Motor adaptation displays a structure-learning effect: adaptation to a new perturbation occurs more quickly when the subject has prior exposure to perturbations with related structure. Although this `learning-to-learn' effect is well documented, its underlying computational mechanisms are poorly understood. We present a new model of motor structure learning, approaching it from the point of view of deep reinforcement learning. Previous work outside of motor control has shown how recurrent neural networks can account for learning-to-learn effects. We leverage this insight to address motor learning, by importing it into the setting of model-based reinforcement learning. We apply the resulting processing architecture to empirical findings from a landmark study of structure learning in target-directed reaching (Braun et al., 2009), and discuss its implications for a wider range of learning-to-learn phenomena.Comment: 39th Annual Meeting of the Cognitive Science Society, to appea

Neural correlates of cognitive dissonance and choice-induced preference change

According to many modern economic theories, actions simply reflect an individual's preferences, whereas a psychological phenomenon called “cognitive dissonance” claims that actions can also create preference. Cognitive dissonance theory states that after making a difficult choice between two equally preferred items, the act of rejecting a favorite item induces an uncomfortable feeling (cognitive dissonance), which in turn motivates individuals to change their preferences to match their prior decision (i.e., reducing preference for rejected items). Recently, however, Chen and Risen [Chen K, Risen J (2010) J Pers Soc Psychol 99:573–594] pointed out a serious methodological problem, which casts a doubt on the very existence of this choice-induced preference change as studied over the past 50 y. Here, using a proper control condition and two measures of preferences (self-report and brain activity), we found that the mere act of making a choice can change self-report preference as well as its neural representation (i.e., striatum activity), thus providing strong evidence for choice-induced preference change. Furthermore, our data indicate that the anterior cingulate cortex and dorsolateral prefrontal cortex tracked the degree of cognitive dissonance on a trial-by-trial basis. Our findings provide important insights into the neural basis of how actions can alter an individual's preferences

Higher media multi-tasking activity is associated with smaller gray-matter density in the anterior cingulate cortex

Media multitasking, or the concurrent consumption of multiple media forms, is increasingly prevalent in today’s society and has been associated with negative psychosocial and cognitive impacts. Individuals who engage in heavier media-multitasking are found to perform worse on cognitive control tasks and exhibit more socio-emotional difficulties. However, the neural processes associated with media multi-tasking remain unexplored. The present study investigated relationships between media multitasking activity and brain structure. Research has demonstrated that brain structure can be altered upon prolonged exposure to novel environments and experience. Thus, we expected differential engagements in media multitasking to correlate with brain structure variability. This was confirmed via Voxel-Based Morphometry (VBM) analyses: Individuals with higher Media Multitasking Index (MMI) scores had smaller gray matter density in the anterior cingulate cortex (ACC). Functional connectivity between this ACC region and the precuneus was negatively associated with MMI. Our findings suggest a possible structural correlate for the observed decreased cognitive control performance and socio-emotional regulation in heavy media-multitaskers. While the cross-sectional nature of our study does not allow us to specify the direction of causality, our results brought to light novel associations between individual media multitasking behaviors and ACC structure differences

Evidence for surprise minimization over value maximization in choice behavior

Classical economic models are predicated on the idea that the ultimate aim of choice is to maximize utility or reward. In contrast, an alternative perspective highlights the fact that adaptive behavior requires agents' to model their environment and minimize surprise about the states they frequent. We propose that choice behavior can be more accurately accounted for by surprise minimization compared to reward or utility maximization alone. Minimizing surprise makes a prediction at variance with expected utility models; namely, that in addition to attaining valuable states, agents attempt to maximize the entropy over outcomes and thus 'keep their options open'. We tested this prediction using a simple binary choice paradigm and show that human decision-making is better explained by surprise minimization compared to utility maximization. Furthermore, we replicated this entropy-seeking behavior in a control task with no explicit utilities. These findings highlight a limitation of purely economic motivations in explaining choice behavior and instead emphasize the importance of belief-based motivations

Design for a Darwinian Brain: Part 1. Philosophy and Neuroscience

Physical symbol systems are needed for open-ended cognition. A good way to understand physical symbol systems is by comparison of thought to chemistry. Both have systematicity, productivity and compositionality. The state of the art in cognitive architectures for open-ended cognition is critically assessed. I conclude that a cognitive architecture that evolves symbol structures in the brain is a promising candidate to explain open-ended cognition. Part 2 of the paper presents such a cognitive architecture.Comment: Darwinian Neurodynamics. Submitted as a two part paper to Living Machines 2013 Natural History Museum, Londo

Defecting or not defecting: how to "read" human behavior during cooperative games by EEG measurements

Understanding the neural mechanisms responsible for human social interactions is difficult, since the brain activities of two or more individuals have to be examined simultaneously and correlated with the observed social patterns. We introduce the concept of hyper-brain network, a connectivity pattern representing at once the information flow among the cortical regions of a single brain as well as the relations among the areas of two distinct brains. Graph analysis of hyper-brain networks constructed from the EEG scanning of 26 couples of individuals playing the Iterated Prisoner's Dilemma reveals the possibility to predict non-cooperative interactions during the decision-making phase. The hyper-brain networks of two-defector couples have significantly less inter-brain links and overall higher modularity - i.e. the tendency to form two separate subgraphs - than couples playing cooperative or tit-for-tat strategies. The decision to defect can be "read" in advance by evaluating the changes of connectivity pattern in the hyper-brain network