Trommershäuser, Julia

English

PubMed

Julia Trommershäuser

Frontiers - Publisher Connector

concerned with the discounting of future rewards, as opposed to 
smaller rewards that may be preferred when available immediately 
(Ray and Bossaerts, 2011).
Bayesian decision theory describes how to select between pos-
sible courses of action on the basis of a specified loss function, e.g., 
expected utility, in many circumstances (Pezzulo and Rigoli, 2011).
However, during most decision tasks, neither the outcomes 
associated with different plans of actions, nor the probability of 
their occurrence is available to the decision-maker prior to mak-
ing the decision. Under these conditions, it is necessary to learn 
about the available outcomes from trial-and-error experience 
(Stoloff et al., 2011). The field of reinforcement learning (e.g., 
Sutton and Barto, 1998; Balleine et al., 2008; Niv and Montague, 
2008) extends decision-theoretic accounts to situations involving 
learning. This theoretical framework, and its underlying statis-
tical principles, have been used to explain the role of learning 
both in traditional choice tasks (e.g., Behrens et al., 2007; Dayan 
and Daw, 2008), and in sensorimotor adaptation (e.g., Körding 
et al., 2007).
Reinforcement learning theories also play an important role in 
another key area of current work in decision-making: the study 
of the neural processes underlying these functions. Notably, this 
system is involved both in motivated decisions (Kurniawan et al., 
2011) and in movement, though how these functions relate is a 
subject of ongoing research and controversy. In addition to dopa-
minergic recordings, monkey work on learning about decisions 
from rewards has focused on frontal cortex (e.g., Lee and Seo, 2007) 
and also posterior parietal cortex, which is classically thought to 
be involved in the so-called dorsal visual processing stream.
Besides the underlying theoretical parallels between the two 
fields, and the growing interest in both fields in similar learning 
processes and common neural mechanisms, two recent develop-
ments make the time ripe to begin building a bridge between 
research on decision-making and the research on optimal motor 
control. The first is the availability of new experimental tools such 
as functional MRI to assess and measure the neural processes 
underlying human and non-human decision behavior, during 
the decision process and following choice (Hansen et al., 2011; 
Santos et al., 2011). The second are new analytical tools, specifi-
cally the growing application of behavioral and computational 
methods from psychophysics and Bayesian decision theory in 
the context of decision-making (Baldassi and Simoncini, 2011). 
This has created a situation in which researchers across fields 
have started to use a common set of conceptual tools for defining 
problems, building computational models, and designing and 
analyzing experiments.
Research on economic decision-making seeks to understand 
how subjects choose between plans of action (lotteries, gam-
bles, prospects) that have economic consequences. The key 
difficulty in making such decisions is that typically no plan 
of action available to the decision-maker guarantees a specific 
outcome, rather, consequences are risky or uncertain. More 
recently, researchers in psychology, behavioral and computa-
tional neuroscience, and psychology have started to apply these 
theoretical principles to studying choice behavior and its neu-
ral basis in the laboratory, for instance in electrophysiological 
studies of animals making choices for primary reward such as 
juice, and neuroimaging studies of humans making choices 
for money. Moreover, researchers across all these fields are, in 
parallel, studying how decisions are guided by learning and 
how the computations relevant to decisions and choices are 
represented neurally.
This Frontiers Research Topic on The Neurobiology of Choice 
combines contributions from researchers from the fields of neuro-
biology, behavioral, and computational neuroscience that discuss 
the neural computations underlying decision-making and adaptive 
behavior.
Placing motor and cognitive decisions in a common theoreti-
cal framework brings into sharp relief one apparent difference 
between them. Researchers have long argued that humans and 
animals mostly make choices in sensorimotor tasks that are nearly 
 optimal – in the sense of approaching maximal expected utility – or 
complying with principles of statistical inference. In contrast, work 
in traditional economic decision-making tasks often focuses on 
situations in which participants violate the predictions of expected 
utility theory, for instance by misrepresenting the frequency of 
rare events or due to interference with emotional factors (Kirk 
et al., 2011).
More recently, researchers in psychology and neuroscience have 
started to apply these theoretical principles to studying choice 
behavior and its neural basis in the laboratory, for instance in elec-
trophysiological studies of animals making choices for primary 
reward such as juice (Milstein and Dorris, 2011; Opris et al., 2011) 
and neuroimaging studies of humans making choices for money 
(Delgado et al., 2011).
Meanwhile, a largely different group of researchers, working 
in the field of sensorimotor control, have also recently drawn on 
statistical decision theory and reinforcement learning in order 
to reformulate the problem of hand and eye movement control 
(Stoloff et al., 2011). An important area of current research in 
both areas is how decisions are impacted by learning (Delgado 
et al., 2011) and when reward is delayed. The latter type of task is 
Frontiers research topic on the neurobiology of choice
Julia Trommershäuser*
Center for Neural Science, New York University, New York, NY, USA
*Correspondence: julia.trommershaeuser@nyu.edu
www.frontiersin.org October 2011 | Volume 5 | Article 119 | 1
Editorial
published: 10 October 2011
doi: 10.3389/fnins.2011.00119
Stoloff, R. H., Taylor, J. A., Xu, J., 
Ridderikhoff, A., and Ivry, R. B. 
(2011). Effect of reinforcement history 
on hand choice in an unconstrained 
reaching task. Front. Neurosci. 5:41. 
doi: 10.3389/fnins.2011.00041
Sutton, R. S., and Barto, A. G. (1998). 
Reinforcement  Lear ning :  An 
Introduction. Cambridge, MA: MIT 
Press.
Received: 15 September 2011; accepted: 
15 September 2011; published online: 10 
October 2011.
Citation: Trommershäuser J (2011) 
Frontiers research topic on the neurobiol-
ogy of choice. Front. Neurosci. 5:119. doi: 
10.3389/fnins.2011.00119
This article was submitted to Frontiers 
in Decision Neuroscience, a specialty of 
Frontiers in Neuroscience.
Copyright © 2011 Trommershäuser. This 
is an open-access article subject to a non-
exclusive license between the authors and 
Frontiers Media SA, which permits use, dis-
tribution and reproduction in other forums, 
provided the original authors and source are 
credited and other Frontiers conditions are 
complied with.
ditions of uncertainty. Front. Neurosci. 
in press.
Niv, Y., and Montague, P. R. (2008). 
“Theoretical and empirical studies of 
learning,” in Neuroeconomics: Decision 
Making and the Brain, eds P. W. Glimcher, 
C. F. Camerer, E. Fehr, and R. A. Poldrack 
(London: Elsevier), 331–348.
Opris, I., Lebedev, M., and Nelson, R. 
J. (2011). Motor planning under 
unpredictable reward: modulations 
of movement vigor and primate stria-
tum activity. Front. Neurosci. 5:61. doi: 
10.3389/fnins.2011.00061
Pezzulo, G., and Rigoli, F.  (2011).  The value 
of foresight: how prospection affects 
decision-making. Front. Neurosci. 5:79. 
doi: 10.3389/fnins.2011.00079
Ray, D., and Bossaerts, P. (2011). Positive 
temporal dependence of the biologi-
cal clock implies hyperbolic discount-
ing. Front. Neurosci. 5:2. doi: 10.3389/
fnins.2011.00002
Santos, J. P., Seixas, D., Brandão, S., and 
Moutinho, L. (2011). Investigating 
the role of the ventromedial pre-
frontal cortex in the assessment of 
brands. Front. Neurosci. 5:77. doi: 
10.3389/fnins.2011.00077
Hansen, K. A., Hillenbrand, S. F., and 
Ungerleider, L. G. (2011). Persistency of 
priors-induced bias in decision behav-
ior and the fMRI signal. Front. Neurosci. 
5:29. doi: 10.3389/fnins.2011.00029
Kirk, U., Downar, J., and Montague, P. R. 
(2011). Interoception drives increased 
rational decision-making in medita-
tors playing the ultimatum game. 
Front. Neurosci. 5:49. doi: 10.3389/
fnins.2011.00049
Körding, K. P., Tenenbaum, J. B., and 
Shadmehr, R. (2007). The dynamics of 
memory as a consequence of optimal 
adaptation to a changing body. Nat. 
Neurosci. 10, 779–786.
Kurniawan, I. T., Guitart-Masip, M., 
and Dolan, R. J. (2011). Dopamine 
and effort-based decision making. 
Front. Neurosci. 5:81. doi: 10.3389/
fnins.2011.00081
Lee, D., and Seo, H. (2007). Mechanisms of 
reinforcement learning and decision 
making in the primate  dorsolateral 
prefrontal cortex. Ann. N. Y. Acad. 
Sci. 1104, 108–122.
Milstein, D. M., and Dorris, M. C. (2011). 
Saccade generation is influenced by rela-
tive expected subjective value under con-
RefeRences
Baldassi, S., and Simoncini, C. (2011). 
Reward sharpens orientation cod-
ing independently of attention. 
Front. Neurosci. 5:13. doi: 10.3389/
fnins.2011.00013
Balleine, B. W., Daw, N. D., and Doherty, 
J. P. (2008). “Multiple forms of value 
learning and the function of dopa-
mine,” in Neuroeconomics: Decision 
Making and the Brain, eds P. W. 
Glimcher, C. F. Camerer, E. Fehr, and 
R. A. Poldrack (London: Elsevier), 
367–385.
Behrens, T. E., Woolrick, M. W., Walton, 
M. E., and Rushworth, M. F. (2007). 
Learning the value of information in 
an uncertain world. Nat. Neurosci. 10, 
1214–1221.
Dayan, P., and Daw, N. D. (2008). Decision 
theory, reinforcement learning, and 
the brain. Cogn. Affect. Behav. Neurosci. 
8, 429–453.
Delgado, M. R., Jou, R. L., and Phelps, E. 
A. (2011). Neural systems underlying 
aversive conditioning in humans with 
primary and secondary reinforcers. 
Front. Neurosci. 5:71. doi: 10.3389/
fnins.2011.00071
Frontiers in Neuroscience | Decision Neuroscience  October 2011 | Volume 5 | Article 119 | 2
Trommershäuser Research topic: neurobiology of choice


Frontiers research topic on the neurobiology of choice

Crossref

Frontiers in Neuroscience

Julia eTrommershaeuser

Directory of Open Access Journals

Frontiers Research Topic on The Neurobiology of Choice

http://doaj.org/search?source=%7B%22query%22%3A%7B%22bool%22%3A%7B%22must%22%3A%5B%7B%22term%22%3A%7B%22id%22%3A%226c38accd005940ff8d7ee7c4e9531234%22%7D%7D%5D%7D%7D%7D

Frontiers Research Topic on the Neurobiology of Choice

Frontiers Research Topic on the Neurobiology of Choice

Abstract

Similar works

Full text

Available Versions

Frontiers - Publisher Connector

Crossref

Directory of Open Access Journals