Building Bridges between Perceptual and Economic Decision-Making: Neural and Computational Mechanisms

Investigation into the neural and computational bases of decision-making has proceeded in two parallel but distinct streams. Perceptual decision-making (PDM) is concerned with how observers detect, discriminate, and categorize noisy sensory information. Economic decision-making (EDM) explores how options are selected on the basis of their reinforcement history. Traditionally, the sub-fields of PDM and EDM have employed different paradigms, proposed different mechanistic models, explored different brain regions, disagreed about whether decisions approach optimality. Nevertheless, we argue that there is a common framework for understanding decisions made in both tasks, under which an agent has to combine sensory information (what is the stimulus) with value information (what is it worth). We review computational models of the decision process typically used in PDM, based around the idea that decisions involve a serial integration of evidence, and assess their applicability to decisions between good and gambles. Subsequently, we consider the contribution of three key brain regions – the parietal cortex, the basal ganglia, and the orbitofrontal cortex (OFC) – to perceptual and EDM, with a focus on the mechanisms by which sensory and reward information are integrated during choice. We find that although the parietal cortex is often implicated in the integration of sensory evidence, there is evidence for its role in encoding the expected value of a decision. Similarly, although much research has emphasized the role of the striatum and OFC in value-guided choices, they may play an important role in categorization of perceptual information. In conclusion, we consider how findings from the two fields might be brought together, in order to move toward a general framework for understanding decision-making in humans and other primates

Metacognitive control of categorial neurobehavioral decision systems

The competing neuro-behavioral decision systems (CNDS) model proposes that the degree to which an individual discounts the future is a function of the relative hyperactivity of an impulsive system based on the limbic and paralimbic brain regions and the relative hypoactivity of an executive system based in prefrontal cortex (PFC). The model depicts the relationship between these categorial systems in terms of the antipodal neurophysiological, behavioral, and decision (cognitive) functions that engender normal and addictive responding. However, a case may be made for construing several components of the impulsive and executive systems depicted in the model as categories (elements) of additional systems that are concerned with the metacognitive control of behavior. Hence, this paper proposes a category-based structure for understanding the effects on behavior of CNDS, which includes not only the impulsive and executive systems of the basic model but a superordinate level of reflective or rational decision-making. Following recent developments in the modeling of cognitive control which contrasts Type 1 (rapid, autonomous, parallel) processing with Type 2 (slower, computationally demanding, sequential) processing, the proposed model incorporates an arena in which the potentially conflicting imperatives of impulsive and executive systems are examined and from which a more appropriate behavioral response than impulsive choice emerges. This configuration suggests a forum in which the interaction of picoeconomic interests, which provide a cognitive dimension for CNDS, can be conceptualized. This proposition is examined in light of the resolution of conflict by means of bundling

Psychoneural reduction: a perspective from neural circuits

Abstract: Psychoneural reduction has been debated extensively in the philosophy of neuroscience. In this article I will evaluate metascientific approaches that claim direct molecular and cellular explanations of cognitive functions. I will initially consider the issues involved in linking cellular properties to behaviour from the general perspective of neural circuits. These circuits that integrate the molecular and cellular components underlying cognition and behaviour, making consideration of circuit properties relevant to reductionist debates. I will then apply this general perspective to specific systems where psychoneural reduction has been claimed, namely hippocampal long-term potentiation and the Aplysia gill-withdrawal reflex

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field

Competing goals draw attention to effort, which then enters cost-benefit computations as input

Different to Kurzban et al., we conceptualize the experience of mental effort as the subjective costs of goal pursuit (i.e., the amount of invested resources relative to the amount of available resources). Rather than being an output of computations that compare costs and benefits of the target and competing goals, effort enters these computations as an inpu

Opportunity cost calculations only determine justified effort-Or, What happened to the resource conservation principle?

We welcome the development of a new model on effort and performance and the critique on existing resource-based models. However, considering the vast evidence for the significant impact of experienced task demand on resource allocation, we conclude that Kurzban et al.'s opportunity cost model is only valid for one performance condition: if task demand is unknown or unspecifie

The Social Cognitive Actor

Multi-Agent Simulation (MAS) of organisations is a methodology that is adopted in this dissertation in order to study and understand human behaviour in organisations. The aim of the research is to design and implementat a cognitive and social multi-agent simulation model based on a selection of social and cognitive theories to fulfill the need for a complex cognitive and social model. The emphasis of this dissertation is the relationship between behaviour of individuals (micro-level) in an organisation and the behaviour of the organisation as a whole (macro-level)

Neural correlates of affordance competition in dorsal premotor cortex

Le travail présenté dans cette thèse porte sur le rôle du cortex prémoteur dorsal (PMd) au sujet de la prise de décision (sélection d’une action parmis nombreux choix) et l'orientation visuelle des mouvements du bras. L’ouvrage décrit des expériences électrophysiologiques chez le singe éveillé (Macaca mulatta) permettant d’adresser une fraction importante des prédictions proposées par l'hypothèse des affordances concurrentes (Cisek, 2006; Cisek, 2007a). Cette hypothèse suggère que le choix de toute action est l’issue d'une concurrence entre les représentations internes des exigences et des atouts de chacune des options présentées (affordances; Gibson, 1979). Un intérêt particulier est donné au traitement de l'information spatiale et la valeur des options (expected value, EV) dans la prise de décisions. La première étude (article 1) explore la façon dont PMd reflète ces deux paramètres dans la période délai ainsi que de leur intéraction. La deuxième étude (article 2) explore le mécanisme de décision de façon plus détaillée et étend les résultats au cortex prémoteur ventral (PMv). Cette étude porte également sur la représentation spatiale et l’EV dans une perspective d'apprentissage. Dans un environnement nouveau les paramètres spatiaux des actions semblent être présents en tout temps dans PMd, malgré que la représentation de l’EV apparaît uniquement lorsque les animaux commencent à prendre des décisions éclairées au sujet de la valeur des options disponibles. La troisième étude (article 3) explore la façon dont PMd est impliqué aux “changements d'esprit“ dans un procès de décision. Cette étude décrit comment la sélection d’une action est mise à jour à la suite d'une instruction de mouvement (GO signal). I II Les résultats principaux des études sont reproduits par un modèle computationnel (Cisek, 2006) suggérant que la prise de décision entre plusieurs actions alternatives peux se faire par voie d’un mécanisme de concurrence (biased competition) qui aurait lieu dans la même région qui spécifie les actions.This thesis examines the role of the dorsal premotor cortex (PMd) in the process of decision making (action selection) and visual guidance of arm movements. The work describes electrophysiological experiments conducted in awake monkeys (Macaca mulatta) and tests a number of important predictions suggested by the affordance competition hypothesis (Cisek, 2006; Cisek, 2007a). This hypothesis suggests that decisions can be viewed as the result of a competition between internal representations of conflicting demands and opportunities for actions or affordances (Gibson, 1979). Specific interest is given to the interaction between spatial information and expected value (EV) in a proposed affordance competition mechanism for action selection. The first study presented (article 1) explores how EV is represented during the delay period in PMd. This study also describes how this area reflects the spatial metrics of the options and examines the interaction between value and spatial information. The second study (article 2) explores the mechanism of action selection in more detail and extends the results to ventral premotor cortex (PMv). This study also addresses the nature of value and spatial representations from a learning perspective. In a novel environment the spatial metrics of the actions seem to be invariably present in PMd, meanwhile EV representations appear only once the animals make behaviorally informed decisions about the value of the available options. The third study (article 3) explores how PMd is involved in “changes of mind” in which action selection is updated following a movement instruction (GO signal). III IV The major findings in all these studies are reproduced by a computational model (Cisek, 2006) suggesting that decisions between actions can be made through a biased competition process that takes place in the same region that specifies the actions

Neuropsychology of reinforcement processes in the rat

This thesis investigated the role played by regions of the prefrontal cortex and ventral striatum in the control of rats’ behaviour by Pavlovian conditioned stimuli, and in their capacity to choose delayed reinforcement. First, the function of the anterior cingulate cortex (ACC) in simple Pavlovian conditioning tasks was addressed. The ACC is a subdivision of prefrontal cortex that has previously been suggested to be critical for the formation of stimulus–reward associations. It was found that lesions of the ACC did not prevent rats from learning a simple conditioned approach response to a conditioned stimulus (CS) predictive of food reward, or from utilizing that CS as a conditioned reinforcer subsequently. Additionally, these subjects successfully acquired a conditioned freezing response to a CS predicting footshock. However, the same animals were impaired at the acquisition of autoshaped behaviour, an impairment that has been demonstrated previously. An autoshaping deficit was also observed when lesions were made following training. The phenomenon of Pavlovian–instrumental transfer was intact in these subjects. The hypothesis was developed that the ACC is not critical for the formation of stimulus–reward associations per se, but is critical when multiple stimuli must be discriminated on the basis of their differential association with reward. In support of this hypothesis, animals with lesions of the ACC were impaired on a version of the conditioned approach task in which a second, neutral stimulus, perceptually similar to the CS, was added; the lesioned subjects exhibited reduced discrimination. Second, the role of the nucleus accumbens (Acb) in Pavlovian–instrumental transfer was investigated. The nucleus accumbens core, together with a larger amygdalar–striatal network of which it is a component, has previously been shown to be necessary for the expression of ‘simple’ Pavlovian–instrumental transfer. Rats with lesions of the nucleus accumbens core (AcbC) and shell (AcbSh) were tested on a ‘response-specific’ Pavlovian–instrumental transfer task, in which a Pavlovian CS selectively enhances instrumental responding for the outcome with which the CS was originally paired. AcbC lesions impaired the response specificity of this effect, while AcbSh lesions abolished Pavlovian–instrumental transfer entirely. These results are consistent with some — but not all — previous results in suggesting that the shell provides ‘vigour’ and the core provides ‘direction’ for the potentiation of behaviour by Pavlovian CSs. Third, an attempt was made to train rats on a task for assessing preference for delayed reinforcement, using the ‘adjusting-delay’ paradigm. It was not immediately apparent that the rats reacted to the contingencies operative in this task, and mathematical analysis of their behaviour was conducted to establish whether their behaviour was sensitive to the delay, and what ‘molar’ features of performance on this task could be explained by delay-independent processes. Fourth, a different delayed reinforcement choice task was developed, modifying a previously published task in which the subject is repeatedly offered a choice, in discrete trials, of a small reward delivered immediately, and a large reward delivered after a delay, with the delays systematically varied by the experimenter. Rats were trained on versions of this task in which the large, delayed reinforcer was or was not explicitly signalled by a cue present during the delay. The behavioural basis of performance on this task was examined, and d-amphetamine, chlordiazepoxide, and alpha-flupenthixol were administered systemically. It was found that the effects of d-amphetamine depended on whether the delayed reinforcer was signalled or unsignalled, increasing preference for signalled delayed reinforcement at some doses, but decreasing preference for unsignalled delayed reinforcement. These results may resolve contradictions in the literature, and are suggested to reflect the known effect of amphetamine to potentiate responding for conditioned reinforcers. Fifth, rats that had been trained on this task (with no explicit signals present during the delay) were given lesions of the ACC, AcbC, or medial prefrontal cortex (mPFC). ACC-lesioned rats were no different from sham-operated controls in their ability to choose a large, delayed reinforcer. Lesions of mPFC reduced the tendency of subjects to shift from one lever to the other during the course of a session, but mPFC-lesioned subjects responded normally to removal of the delays, suggesting a loss of stimulus control. However, rats with lesions of the AcbC were severely impaired on this task, preferring the small, immediate reward, even though they discriminated the reinforcers. Additionally, the effects of intra-Acb amphetamine were assessed using a different version of the delayed reinforcement choice task, and found to have slight but inconsistent effects to reduce preference for the delayed reinforcer, though this effect did not depend on whether the delayed reward was signalled or unsignalled. These results suggest that the AcbC contributes significantly to the rat’s ability to choose a delayed reward, a finding that has important implications for the understanding of Acb function. It is suggested that dysfunction of the AcbC may be a key element in the pathology of impulsivity.Supported by a UK Medical Research Council (MRC) research studentship, 1997–2000, and a James Baird award, University of Cambridge School of Clinical Medicine, 1997–2000