Goal-Directed Decision Making with Spiking Neurons.

UNLABELLED: Behavioral and neuroscientific data on reward-based decision making point to a fundamental distinction between habitual and goal-directed action selection. The formation of habits, which requires simple updating of cached values, has been studied in great detail, and the reward prediction error theory of dopamine function has enjoyed prominent success in accounting for its neural bases. In contrast, the neural circuit mechanisms of goal-directed decision making, requiring extended iterative computations to estimate values online, are still unknown. Here we present a spiking neural network that provably solves the difficult online value estimation problem underlying goal-directed decision making in a near-optimal way and reproduces behavioral as well as neurophysiological experimental data on tasks ranging from simple binary choice to sequential decision making. Our model uses local plasticity rules to learn the synaptic weights of a simple neural network to achieve optimal performance and solves one-step decision-making tasks, commonly considered in neuroeconomics, as well as more challenging sequential decision-making tasks within 1 s. These decision times, and their parametric dependence on task parameters, as well as the final choice probabilities match behavioral data, whereas the evolution of neural activities in the network closely mimics neural responses recorded in frontal cortices during the execution of such tasks. Our theory provides a principled framework to understand the neural underpinning of goal-directed decision making and makes novel predictions for sequential decision-making tasks with multiple rewards. SIGNIFICANCE STATEMENT: Goal-directed actions requiring prospective planning pervade decision making, but their circuit-level mechanisms remain elusive. We show how a model circuit of biologically realistic spiking neurons can solve this computationally challenging problem in a novel way. The synaptic weights of our network can be learned using local plasticity rules such that its dynamics devise a near-optimal plan of action. By systematically comparing our model results to experimental data, we show that it reproduces behavioral decision times and choice probabilities as well as neural responses in a rich set of tasks. Our results thus offer the first biologically realistic account for complex goal-directed decision making at a computational, algorithmic, and implementational level.This research was supported by the Swiss National Science Foundation (J.F., Grant PBBEP3 146112) and the Wellcome Trust (J.F. and M.L.).This is the author accepted manuscript. It is currently under an indefinite embargo pending publication by the Society for Neuroscience

The dynamics of deferred decision

AbstractDecision makers are often unable to choose between the options that they are offered. In these settings they typically defer their decision, that is, delay the decision to a later point in time or avoid the decision altogether. In this paper, we outline eight behavioral findings regarding the causes and consequences of choice deferral that cognitive theories of decision making should be able to capture. We show that these findings can be accounted for by a deferral-based time limit applied to existing sequential sampling models of preferential choice. Our approach to modeling deferral as a time limit in a sequential sampling model also makes a number of novel predictions regarding the interactions between choice probabilities, deferral probabilities, and decision times, and we confirm these predictions in an experiment. Choice deferral is a key feature of everyday decision making, and our paper illustrates how established theoretical approaches can be used to understand the cognitive underpinnings of this important behavioral phenomenon

The cognitive and neural dynamics of memory-based decisions

The recent years have seen the rise of neuroeconomics, a scientific discipline investigating the cognitive and neural principles of value-based decision making. While neuroeconomists made significant progress in characterizing basic computations of value-based decision making, the critical role of memory has all-too-often been neglected. Within this cumulative dissertation thesis, I present four manuscripts, which address the relation of memory and decision making. Manuscript 1 reviews empirical evidence which demonstrates that memory-based decisions are biased in favor of choice options which can be recalled from memory. Adopting cognitive process models, Manuscript 2 demonstrates that this memory bias is rather due to a single decision process, as compared to a dual-process account of memory-based decisions. Manuscript 3 focuses on the temporal dynamics of memory retrieval and choice formation, outlining altered evidence accumulation dynamics of memory-based versus standard value-based decisions. Finally, Manuscript 4 takes the first steps toward a cognitive process model which accounts for the temporal dynamics of both, memory retrieval and decision making. While every manuscript can be approached individually, the synopsis part of this dissertation thesis discusses them in a broader perspective, drawing on the neuroeconomic framework by Rangel et al. (2008). All in all, this dissertation thesis advocates for neuroeconomics to take memory processes more seriously. Future research will especially profit from a deeper understanding of the temporal dynamics of memory retrieval and its relation to decision making

Sequential Sampling Models of Choice: Some Recent Advances

Choice models in marketing and economics are generally derived without specifying the underlying cognitive process of decision making. This approach has been successfully used to predict choice behavior. However, it has not much to say about such aspects of decision making as deliberation, attention, conflict, and cognitive limitations and how these influence choices. In contrast, sequential sampling models developed in cognitive psychology explain observed choices based on assumptions about cognitive processes that return the observed choice as the terminal state. We illustrate three advantages of this perspective. First, making explicit assumptions about underlying cognitive processes results in measures of deliberation, attention, conflict, and cognitive limitation. Second, the mathematical representations of underlying cognitive processes imply well documented departures from Luce’s Choice Axiom such as the similarity, compromise, and attraction effects. Third, the process perspective predicts response time and thus allows for inference based on observed choices and response times. Finally, we briefly discuss the relationship between these cognitive models and rules for statistically optimal decisions in sequential designs

The Neural Computations In The Caudate Nucleus For Reward-Biased Perceptual Decision-Making

Decision-making is a complex process in which our brain has to combine different sources of information, such as noisy sensory evidence and expected reward, in appropriate ways to obtain the outcome that satisfies the decision-maker. Despite various studies on perceptual decision-making and value-based decision making, it is still unclear how the brain combines sensory and reward information to make a complex decision. A prime candidate for mediating this process is the basal ganglia pathway. This pathway is known to make separate contributions to perceptual decisions based on the interpretation of uncertain sensory evidence and value-based decisions that select among outcome options. To begin to investigate what computations are performed by the brain, particularly in the basal ganglia, we trained monkeys to perform a reward-biased visual motion direction discrimination task and performed single-unit extracellular recordings in the caudate nucleus, the input station in the basal ganglia. Fitting the monkeys’ behaviors to a drift-diffusion model, we found that the monkeys used a rational heuristic to combine sensory and reward information. This heuristic is suboptimal but leads to good-enough outcomes. We also found that the monkeys’ reward biases were sensitive to the changes in the reward functions from session to session. This adaptive adjustment could be a possible reason underlying the individual variability in their decision strategies. By recording in the caudate nucleus, we found that it is involved in both the decision-formation and evaluation: before the monkey started accumulating sensory evidence, the caudate neurons represented the reward context that could be used to form a reward bias; during decision-formation, some caudate neurons jointly represented sensory evidence and reward information, which could facilitate the combining of sensory and reward information appropriately. After a decision is made, caudate nucleus represented both decision confidence and reward expectation, two evaluation-related quantities that influence the monkeys’ subsequent decision behaviors