Modelling Emotion Based Reward Valuation with Computational Reinforcement Learning

We show that computational reinforcement learning can model human decision making in the Iowa Gambling Task (IGT). The IGT is a card game, which tests decision making under uncertainty. In our experiments, we found that modulating learning rate decay in Q-learning, enables the approximation of both the behaviour of normal subjects and those who are emotionally impaired by ventromedial prefrontal lesions. Outcomes observed in impaired subjects are modeled by high learning rate decay, while low learning rate decay replicates healthy subjects under otherwise identical conditions. The ventromedial prefrontal cortex has been associated with emotion based reward valuation, and, the value function in reinforcement learning provides an analogous assessment mechanism. Thus reinforcement learning can provide a good model for the role of emotional reward as a modulator of the learning rate

The Emergence of Emotions

Emotion is conscious experience. It is the affective aspect of consciousness. Emotion arises from sensory stimulation and is typically accompanied by physiological and behavioral changes in the body. Hence an emotion is a complex reaction pattern consisting of three components: a physiological component, a behavioral component, and an experiential (conscious) component. The reactions making up an emotion determine what the emotion will be recognized as. Three processes are involved in generating an emotion: (1) identification of the emotional significance of a sensory stimulus, (2) production of an affective state (emotion), and (3) regulation of the affective state. Two opposing systems in the brain (the reward and punishment systems) establish an affective value or valence (stimulus-reinforcement association) for sensory stimulation. This is process (1), the first step in the generation of an emotion. Development of stimulus-reinforcement associations (affective valence) serves as the basis for emotion expression (process 2), conditioned emotion learning acquisition and expression, memory consolidation, reinforcement-expectations, decision-making, coping responses, and social behavior. The amygdala is critical for the representation of stimulus-reinforcement associations (both reward and punishment-based) for these functions. Three distinct and separate architectural and functional areas of the prefrontal cortex (dorsolateral prefrontal cortex, orbitofrontal cortex, anterior cingulate cortex) are involved in the regulation of emotion (process 3). The regulation of emotion by the prefrontal cortex consists of a positive feedback interaction between the prefrontal cortex and the inferior parietal cortex resulting in the nonlinear emergence of emotion. This positive feedback and nonlinear emergence represents a type of working memory (focal attention) by which perception is reorganized and rerepresented, becoming explicit, functional, and conscious. The explicit emotion states arising may be involved in the production of voluntary new or novel intentional (adaptive) behavior, especially social behavior

Deep learning for facial emotion recognition

The ability to perceive and interpret human emotions is an essential as-pect of daily life. The recent success of deep learning (DL) has resulted in the ability to utilize automated emotion recognition by classifying af-fective modalities into a given emotional state. Accordingly, DL has set several state-of-the-art benchmarks on static affective corpora collected in controlled environments. Yet, one of the main limitations of DL based intelligent systems is their inability to generalize on data with nonuniform conditions. For instance, when dealing with images in a real life scenario, where extraneous variables such as natural or artificial lighting are sub-ject to constant change, the resulting changes in the data distribution commonly lead to poor classification performance. These and other con-straints, such as: lack of realistic data, changes in facial pose, and high data complexity and dimensionality increase the difficulty of designing DL models for emotion recognition in unconstrained environments. This thesis investigates the development of deep artificial neural net-work learning algorithms for emotion recognition with specific attention to illumination and facial pose invariance. Moreover, this research looks at the development of illumination and rotation invariant face detection architectures based on deep reinforcement learning. The contributions and novelty of this thesis are presented in the form of several deep learning pose and illumination invariant architectures that offer state-of-the-art classification performance on data with nonuniform conditions. Furthermore, a novel deep reinforcement learning architecture for illumination and rotation invariant face detection is also presented. The originality of this work is derived from a variety of novel deep learning paradigms designed for the training of such architectures

Affect and Learning: a computational analysis

In this thesis we have studied the influence of emotion on learning. We have used computational modelling techniques to do so, more specifically, the reinforcement learning paradigm. Emotion is modelled as artificial affect, a measure that denotes the positiveness versus negativeness of a situation to an artificial agent in a reinforcement learning setting. We have done a range of different experiments to study the effect of affect on learning, including the effect on learning if affect is used to control the exploration behaviour of the agent and the effect on learning when affect is communicated by a human (though real-time analysis of that human__s facial expressions) to a simulated robot. We conclude that affect is a useful concept to consider in adaptive agents that learn based on reinforcement learning and that in some cases affect can indeed help the learning process. Further, affective modelling in this way can help understand the psychological processes that underlie influences of affect on cognition. Finally, we have developed a formal notation for a specific type of emotion theory, i.e., cognitive appraisal theory.UBL - phd migration 201

Maladaptive behavior and affect regulation: A functionalist perspective.

Clinical science has benefited tremendously from taking seriously the proposition that putatively maladaptive behaviors serve psychological functions, prominently among these affect regulation (AR). These functionalist accounts have not only advanced basic clinical science, but also formed the bedrock for the development of effective treatments. Drawing heavily on reinforcement learning theory, we aim to elucidate functional relationships between maladaptive behavior and AR. Specifically, we take the view that maladaptive behaviors are frequently motivated and reinforced by hedonic AR functions (i.e., decreasing negative affect and increasing positive affect) but are also susceptible to becoming stimulus-bound habits. We review empirical evidence related to one such behavior, nonsuicidal self-injury. We close with a brief reflection on future directions. (PsycINFO Database Record (c) 2020 APA, all rights reserved)

Robot pain: a speculative review of its functions

Given the scarce bibliography dealing explicitly with robot pain, this chapter has enriched its review with related research works about robot behaviours and capacities in which pain could play a role. It is shown that all such roles ¿ranging from punishment to intrinsic motivation and planning knowledge¿ can be formulated within the unified framework of reinforcement learning.Peer ReviewedPostprint (author's final draft

The emotional gatekeeper: a computational model of attentional selection and suppression through the pathway from the amygdala to the inhibitory thalamic reticular nucleus

In a complex environment that contains both opportunities and threats, it is important for an organism to flexibly direct attention based on current events and prior plans. The amygdala, the hub of the brain's emotional system, is involved in forming and signaling affective associations between stimuli and their consequences. The inhibitory thalamic reticular nucleus (TRN) is a hub of the attentional system that gates thalamo-cortical signaling. In the primate brain, a recently discovered pathway from the amygdala sends robust projections to TRN. Here we used computational modeling to demonstrate how the amygdala-TRN pathway, embedded in a wider neural circuit, can mediate selective attention guided by emotions. Our Emotional Gatekeeper model demonstrates how this circuit enables focused top-down, and flexible bottom-up, allocation of attention. The model suggests that the amygdala-TRN projection can serve as a unique mechanism for emotion-guided selection of signals sent to cortex for further processing. This inhibitory selection mechanism can mediate a powerful affective 'framing' effect that may lead to biased decision-making in highly charged emotional situations. The model also supports the idea that the amygdala can serve as a relevance detection system. Further, the model demonstrates how abnormal top-down drive and dysregulated local inhibition in the amygdala and in the cortex can contribute to the attentional symptoms that accompany several neuropsychiatric disorders.R01MH057414 - NIMH NIH HHS; R01 MH057414 - NIMH NIH HHS; R01 MH101209 - NIMH NIH HHS; R01NS024760 - NINDS NIH HHS; R01MH101209 - NIMH NIH HHS; R01 NS024760 - NINDS NIH HH