The fundamental problem of command : plan and compliance in a partially centralised economy

When a principal gives an order to an agent and advances resources for its implementation, the temptations for the agent to shirk or steal from the principal rather than comply constitute the fundamental problem of command. Historically, partially centralised command economies enforced compliance in various ways, assisted by nesting the fundamental problem of exchange within that of command. The Soviet economy provides some relevant data. The Soviet command system combined several enforcement mechanisms in an equilibrium that shifted as agents learned and each mechanism's comparative costs and benefits changed. When the conditions for an equilibrium disappeared, the system collapsed.Comparative Economic Studies (2005) 47, 296–314. doi:10.1057/palgrave.ces.810011

Reinforcement learning or active inference?

This paper questions the need for reinforcement learning or control theory when optimising behaviour. We show that it is fairly simple to teach an agent complicated and adaptive behaviours using a free-energy formulation of perception. In this formulation, agents adjust their internal states and sampling of the environment to minimize their free-energy. Such agents learn causal structure in the environment and sample it in an adaptive and self-supervised fashion. This results in behavioural policies that reproduce those optimised by reinforcement learning and dynamic programming. Critically, we do not need to invoke the notion of reward, value or utility. We illustrate these points by solving a benchmark problem in dynamic programming; namely the mountain-car problem, using active perception or inference under the free-energy principle. The ensuing proof-of-concept may be important because the free-energy formulation furnishes a unified account of both action and perception and may speak to a reappraisal of the role of dopamine in the brain

Genetic inhibition of neurotransmission reveals role of glutamatergic input to dopamine neurons in high-effort behavior

Midbrain dopamine neurons are crucial for many behavioral and cognitive functions. As the major excitatory input, glutamatergic afferents are important for control of the activity and plasticity of dopamine neurons. However, the role of glutamatergic input as a whole onto dopamine neurons remains unclear. Here we developed a mouse line in which glutamatergic inputs onto dopamine neurons are specifically impaired, and utilized this genetic model to directly test the role of glutamatergic inputs in dopamine-related functions. We found that while motor coordination and reward learning were largely unchanged, these animals showed prominent deficits in effort-related behavioral tasks. These results provide genetic evidence that glutamatergic transmission onto dopaminergic neurons underlies incentive motivation, a willingness to exert high levels of effort to obtain reinforcers, and have important implications for understanding the normal function of the midbrain dopamine system.Fil: Hutchison, M. A.. National Institutes of Health; Estados UnidosFil: Gu, X.. National Institutes of Health; Estados UnidosFil: Adrover, Martín Federico. National Institutes of Health; Estados Unidos. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Investigaciones en Ingeniería Genética y Biología Molecular "Dr. Héctor N. Torres"; ArgentinaFil: Lee, M. R.. National Institutes of Health; Estados UnidosFil: Hnasko, T. S.. University of California at San Diego; Estados UnidosFil: Alvarez, V. A.. National Institutes of Health; Estados UnidosFil: Lu, W.. National Institutes of Health; Estados Unido

Reward processing in autism: a thematic series

This thematic series presents theoretical and empirical papers focused on understanding autism from the perspective of reward processing deficits. Although the core symptoms of autism have not traditionally been conceptualized with respect to altered reward-based processes, it is clear that brain reward circuitry plays a critical role in guiding social and nonsocial learning and behavior throughout development. Additionally, brain reward circuitry may respond to social sources of information in ways that are similar to responses to primary rewards, and recent clinical data consistently suggest abnormal behavioral and neurobiologic responses to rewards in autism. This thematic series presents empirical data and review papers that highlight the utility of considering autism from the perspective of reward processing deficits. Our hope is that this novel framework may further elucidate autism pathophysiology, with the ultimate goal of yielding novel insights with potential therapeutic implications

Prolonged dopamine signalling in striatum signals proximity and value of distant rewards

Predictions about future rewarding events have a powerful influence on behaviour. The phasic spike activity of dopamine-containing neurons, and corresponding dopamine transients in the striatum, are thought to underlie these predictions, encoding positive and negative reward prediction errors. However, many behaviours are directed towards distant goals, for which transient signals may fail to provide sustained drive. Here we report an extended mode of reward-predictive dopamine signalling in the striatum that emerged as rats moved towards distant goals. These dopamine signals, which were detected with fast-scan cyclic voltammetry (FSCV), gradually increased or—in rare instances—decreased as the animals navigated mazes to reach remote rewards, rather than having phasic or steady tonic profiles. These dopamine increases (ramps) scaled flexibly with both the distance and size of the rewards. During learning, these dopamine signals showed spatial preferences for goals in different locations and readily changed in magnitude to reflect changing values of the distant rewards. Such prolonged dopamine signalling could provide sustained motivational drive, a control mechanism that may be important for normal behaviour and that can be impaired in a range of neurologic and neuropsychiatric disorders.National Institutes of Health (U.S.) (Grant R01 MH060379)National Parkinson Foundation (U.S.)Cure Huntington’s Disease Initiative, Inc. (Grant A-5552)Stanley H. and Sheila G. Sydney Fun

Gender-Related Differences in the Prevalence of Cardiovascular Disease Risk Factors and their Correlates in Urban Tanzania.

\ud Urban areas in Africa suffer a serious problem with dual burden of infectious diseases and emerging chronic diseases such as cardiovascular diseases (CVD) and diabetes which pose a serious threat to population health and health care resources. However in East Africa, there is limited literature in this research area. The objective of this study was to examine the prevalence of cardiovascular disease risk factors and their correlates among adults in Temeke, Dar es Salaam, Tanzania. Results of this study will help inform future research and potential preventive and therapeutic interventions against such chronic diseases. The study design was a cross sectional epidemiological study. A total of 209 participants aged between 44 and 66 years were included in the study. A structured questionnaire was used to evaluate socioeconomic and lifestyle characteristics. Blood samples were collected and analyzed to measure lipid profile and fasting glucose levels. Cardiovascular risk factors were defined using World Health Organization criteria. The age-adjusted prevalence of obesity (BMI > or = 30) was 13% and 35%, among men and women (p = 0.0003), respectively. The prevalence of abdominal obesity was 11% and 58% (p < 0.0001), and high WHR (men: >0.9, women: >0.85) was 51% and 73% (p = 0.002) for men and women respectively. Women had 4.3 times greater odds of obesity (95% CI: 1.9-10.1), 14.2-fold increased odds for abdominal adiposity (95% CI: 5.8-34.6), and 2.8 times greater odds of high waist-hip-ratio (95% CI: 1.4-5.7), compared to men. Women had more than three-fold greater odds of having metabolic syndrome (p = 0.001) compared to male counterparts, including abdominal obesity, low HDL-cholesterol, and high fasting blood glucose components. In contrast, female participants had 50% lower odds of having hypertension, compared to men (95%CI: 0.3-1.0). Among men, BMI and waist circumference were significantly correlated with blood pressure, triglycerides, total, LDL-, and HDL-cholesterol (BMI only), and fasting glucose; in contrast, only blood pressure was positively associated with BMI and waist circumference in women. The prevalence of CVD risk factors was high in this population, particularly among women. Health promotion, primary prevention, and health screening strategies are needed to reduce the burden of cardiovascular disease in Tanzania.\u

Role of the mesoamygdaloid dopamine projection in emotional learning

Amygdala dopamine is crucially involved in the acquisition of Pavlovian associations, as measured via conditioned approach to the location of the unconditioned stimulus (US). However, learning begins before skeletomotor output, so this study assessed whether amygdala dopamine is also involved in earlier 'emotional' learning. A variant of the conditioned reinforcement (CR) procedure was validated where training was restricted to curtail the development of selective conditioned approach to the US location, and effects of amygdala dopamine manipulations before training or later CR testing assessed. Experiment 1a presented a light paired (CS+ group) or unpaired (CS- group) with a US. There were 1, 2 or 10 sessions, 4 trials per session. Then, the US was removed, and two novel levers presented. One lever (CR+) presented the light, and lever pressing was recorded. Experiment 1b also included a tone stimulus. Experiment 2 applied intra-amygdala R(+) 7-OH-DPAT (10 nmol/1.0 A mu l/side) before two training sessions (Experiment 2a) or a CR session (Experiment 2b). For Experiments 1a and 1b, the CS+ group preferred the CR+ lever across all sessions. Conditioned alcove approach during 1 or 2 training sessions or associated CR tests was low and nonspecific. In Experiment 2a, R(+) 7-OH-DPAT before training greatly diminished lever pressing during a subsequent CR test, preferentially on the CR+ lever. For Experiment 2b, R(+) 7-OH-DPAT infusions before the CR test also reduced lever pressing. Manipulations of amygdala dopamine impact the earliest stage of learning in which emotional reactions may be most prevalent

From uncertainty to reward: BOLD characteristics differentiate signaling pathways

<p>Abstract</p> <p>Background</p> <p>Reward value and uncertainty are represented by dopamine neurons in monkeys by distinct phasic and tonic firing rates. Knowledge about the underlying differential dopaminergic pathways is crucial for a better understanding of dopamine-related processes. Using functional magnetic resonance blood-oxygen level dependent (BOLD) imaging we analyzed brain activation in 15 healthy, male subjects performing a gambling task, upon expectation of potential monetary rewards at different reward values and levels of uncertainty.</p> <p>Results</p> <p>Consistent with previous studies, ventral striatal activation was related to both reward magnitudes and values. Activation in medial and lateral orbitofrontal brain areas was best predicted by reward uncertainty. Moreover, late BOLD responses relative to trial onset were due to expectation of different reward values and likely to represent phasic dopaminergic signaling. Early BOLD responses were due to different levels of reward uncertainty and likely to represent tonic dopaminergic signals.</p> <p>Conclusions</p> <p>We conclude that differential dopaminergic signaling as revealed in animal studies is not only represented locally by involvement of distinct brain regions but also by distinct BOLD signal characteristics.</p

The Functional DRD3 Ser9Gly Polymorphism (rs6280) Is Pleiotropic, Affecting Reward as Well as Movement

Abnormalities of motivation and behavior in the context of reward are a fundamental component of addiction and mood disorders. Here we test the effect of a functional missense mutation in the dopamine 3 receptor (DRD3) gene (ser9gly, rs6280) on reward-associated dopamine (DA) release in the striatum. Twenty-six healthy controls (HCs) and 10 unmedicated subjects with major depressive disorder (MDD) completed two positron emission tomography (PET) scans with [11C]raclopride using the bolus plus constant infusion method. On one occasion subjects completed a sensorimotor task (control condition) and on another occasion subjects completed a gambling task (reward condition). A linear regression analysis controlling for age, sex, diagnosis, and self-reported anhedonia indicated that during receipt of unpredictable monetary reward the glycine allele was associated with a greater reduction in D2/3 receptor binding (i.e., increased reward-related DA release) in the middle (anterior) caudate (p<0.01) and the ventral striatum (p<0.05). The possible functional effect of the ser9gly polymorphism on DA release is consistent with previous work demonstrating that the glycine allele yields D3 autoreceptors that have a higher affinity for DA and display more robust intracellular signaling. Preclinical evidence indicates that chronic stress and aversive stimulation induce activation of the DA system, raising the possibility that the glycine allele, by virtue of its facilitatory effect on striatal DA release, increases susceptibility to hyperdopaminergic responses that have previously been associated with stress, addiction, and psychosis