Neural computations of threat in the aftermath of combat trauma

© 2019, The Author(s), under exclusive licence to Springer Nature America, Inc. By combining computational, morphological, and functional analyses, this study relates latent markers of associative threat learning to overt post-traumatic stress disorder (PTSD) symptoms in combat veterans. Using reversal learning, we found that symptomatic veterans showed greater physiological adjustment to cues that did not predict what they had expected, indicating greater sensitivity to prediction errors for negative outcomes. This exaggerated weighting of prediction errors shapes the dynamic learning rate (associability) and value of threat predictive cues. The degree to which the striatum tracked the associability partially mediated the positive correlation between prediction-error weights and PTSD symptoms, suggesting that both increased prediction-error weights and decreased striatal tracking of associability independently contribute to PTSD symptoms. Furthermore, decreased neural tracking of value in the amygdala, in addition to smaller amygdala volume, independently corresponded to higher PTSD symptom severity. These results provide evidence for distinct neurocomputational contributions to PTSD symptoms

Neural computations of threat in the aftermath of combat trauma

By combining computational, morphological, and functional analyses, this study relates latent markers of associative threat learning to overt post-traumatic stress disorder (PTSD) symptoms in combat veterans. Using reversal learning, we found that symptomatic veterans showed greater physiological adjustment to cues that did not predict what they had expected, indicating greater sensitivity to prediction errors for negative outcomes. This exaggerated weighting of prediction errors shapes the dynamic learning rate (associability) and value of threat predictive cues. The degree to which the striatum tracked the associability partially mediated the positive correlation between prediction-error weights and PTSD symptoms, suggesting that both increased prediction-error weights and decreased striatal tracking of associability independently contribute to PTSD symptoms. Furthermore, decreased neural tracking of value in the amygdala, in addition to smaller amygdala volume, independently corresponded to higher PTSD symptom severity. These results provide evidence for distinct neurocomputational contributions to PTSD symptoms

Neural Computations Mediating One-Shot Learning in the Human Brain

Incremental learning, in which new knowledge is acquired gradually through trial and error, can be distinguished from one-shot learning, in which the brain learns rapidly from only a single pairing of a stimulus and a consequence. Very little is known about how the brain transitions between these two fundamentally different forms of learning. Here we test a computational hypothesis that uncertainty about the causal relationship between a stimulus and an outcome induces rapid changes in the rate of learning, which in turn mediates the transition between incremental and one-shot learning. By using a novel behavioral task in combination with functional magnetic resonance imaging (fMRI) data from human volunteers, we found evidence implicating the ventrolateral prefrontal cortex and hippocampus in this process. The hippocampus was selectively “switched” on when one-shot learning was predicted to occur, while the ventrolateral prefrontal cortex was found to encode uncertainty about the causal association, exhibiting increased coupling with the hippocampus for high-learning rates, suggesting this region may act as a “switch,” turning on and off one-shot learning as required

A second look at memory: Different Approaches to Understanding Diversity in Memory and Cognition

Memory lies at the heart of human cognitive abilities. Therefore, understanding it from neural, psychological and computational viewpoints is of key importance for computational neuroscience, psychology and beyond. In this thesis, I explore two prominent, but different, memory systems: episodic memory and working memory. First, I propose a modification to a recent reinforcement learning algorithm for decision making in which single memories of events, i.e., episodic memories, are integrated to compute the long run value of actions. I argue that these memories are recalled and that their contributions are weighted based on context. Further, I propose that predictions made by this algorithm are combined with those that come from a standard, model-free, reinforcement learning algorithm. I suggest that humans can flexibly choose between these two sources of information to make decisions and guide actions. I show that the resulting combined model best fits data on human choices, outperforming previously proposed models. To complement these algorithmic and psychological suggestions, I present a generative model of the world according to which this sort of episodic recall is an appropriate method for making inferences and predictions of future rewards. Contrary to other suggestions for reward-based learning, this generative model can model events that not only drift continuously in time, but can also suddenly change to new or repeated events. Turning to working memory, I use information theoretic analyses to show that dynamic synapses, whose strengths adjust with usage, can increase its capacity. I argue that these components should be included in the study of working memory. The thesis ends with an explanation of the connections between these memory systems

Reinforcement learning approaches to the analysis of the emergence of goal-directed behaviour

Over recent decades, theoretical neuroscience, helped by computational methods such as Reinforcement Learning (RL), has provided detailed descriptions of the psychology and neurobiology of decision-making. RL has provided many insights into the mechanisms underlying decision-making processes from neuronal to behavioral levels. In this work, we attempt to demonstrate the effectiveness of RL methods in explaining behavior in a normative setting through three main case studies. Evidence from literature shows that, apart from the commonly discussed cognitive search process, that governs the solution procedure of a planning task, there is an online perceptual process that directs the action selection towards moves that appear more ‘natural’ at a given configuration of a task. These two processes can be partially dissociated through developmental studies, with perceptual processes apparently more dominant in the planning of younger children, prior to the maturation of executive functions required for the control of search. Therefore, we present a formalization of planning processes to account for perceptual features of the task, and relate it to human data. Although young children are able to demonstrate their preferences by using physical actions, infants are restricted because of their as-yet-undeveloped motor skills. Eye-tracking methods have been employed to tackle this difficulty. Exploring different model-free RL algorithms and their possible cognitive realizations in decision making, in a second case study, we demonstrate behavioral signatures of decision making processes in eye-movement data and provide a potential framework for integrating eye-movement patterns with behavioral patterns. Finally, in a third project we examine how uncertainty in choices might guide exploration in 10-year-olds, using an abstract RL-based mathematical model. Throughout, aspects of action selection are seen as emerging from the RL computational framework. We, thus, conclude that computational descriptions of the developing decision making functions provide one plausible avenue by which to normatively characterize and define the functions that control action selection

A Computational Role for Arousal in Optimal Inference

Making accurate predictions is one of the most critical functions of the brain. Whether made by a monkey deciding where to forage, a deer deciding which way to run, or a wall-street broker deciding how to invest, decisions are informed by expectations about possible future outcomes. These expectations are learned over time through experience and are rapidly adjusted when they fail to match observations. Here I propose and support the thesis that learning systems in the brain optimize the accuracy of predictions in a changing world, even though this necessitates becoming insensitive to incoming sensory information under some conditions. Furthermore I propose a biologically inspired model for achieving accurate predictions and suggest a novel role for the arousal system in optimally adjusting the influence of incoming sensory information. I support these theses with a series of experiments that utilize computational modeling, as well as behavioral and pupillometric measurements in humans