Altruistic Learning

The origin of altruism remains one of the most enduring puzzles of human behaviour. Indeed, true altruism is often thought either not to exist, or to arise merely as a miscalculation of otherwise selfish behaviour. In this paper, we argue that altruism emerges directly from the way in which distinct human decision-making systems learn about rewards. Using insights provided by neurobiological accounts of human decision-making, we suggest that reinforcement learning in game-theoretic social interactions (habitisation over either individuals or games) and observational learning (either imitative of inference based) lead to altruistic behaviour. This arises not only as a result of computational efficiency in the face of processing complexity, but as a direct consequence of optimal inference in the face of uncertainty. Critically, we argue that the fact that evolutionary pressure acts not over the object of learning (‘what’ is learned), but over the learning systems themselves (‘how’ things are learned), enables the evolution of altruism despite the direct threat posed by free-riders

Pain: A Precision Signal for Reinforcement Learning and Control.

Since noxious stimulation usually leads to the perception of pain, pain has traditionally been considered sensory nociception. But its variability and sensitivity to a broad array of cognitive and motivational factors have meant it is commonly viewed as inherently imprecise and intangibly subjective. However, the core function of pain is motivational-to direct both short- and long-term behavior away from harm. Here, we illustrate that a reinforcement learning model of pain offers a mechanistic understanding of how the brain supports this, illustrating the underlying computational architecture of the pain system. Importantly, it explains why pain is tuned by multiple factors and necessarily supported by a distributed network of brain regions, recasting pain as a precise and objectifiable control signal

Response heterogeneity: Challenges for personalised medicine and big data approaches in psychiatry and chronic pain

Response rates to available treatments for psychological and chronic pain disorders are poor, and there is a considerable burden of suffering and disability for patients, who often cycle through several rounds of ineffective treatment. As individuals presenting to the clinic with symptoms of these disorders are likely to be heterogeneous, there is considerable interest in the possibility that different constellations of signs could be used to identify subgroups of patients that might preferentially benefit from particular kinds of treatment. To this end, there has been a recent focus on the application of machine learning methods to attempt to identify sets of predictor variables (demographic, genetic, etc.) that could be used to target individuals towards treatments that are more likely to work for them in the first instance. Importantly, the training of such models generally relies on datasets where groups of individual predictor variables are labelled with a binary outcome category − usually ‘responder’ or ‘non-responder’ (to a particular treatment). However, as previously highlighted in other areas of medicine, there is a basic statistical problem in classifying individuals as ‘responding’ to a particular treatment on the basis of data from conventional randomized controlled trials. Specifically, insufficient information on the partition of variance components in individual symptom changes mean that it is inappropriate to consider data from the active treatment arm alone in this way. This may be particularly problematic in the case of psychiatric and chronic pain symptom data, where both within-subject variability and measurement error are likely to be high. Here, we outline some possible solutions to this problem in terms of dataset design and machine learning methodology, and conclude that it is important to carefully consider the kind of inferences that particular training data are able to afford, especially in arenas where the potential clinical benefit is so large

Decisions about Decisions

A major puzzle of decision making is how the brain decides which decision system to use at any one time. In this issue of Neuron, Lee et al. (2014) provide a theoretical, behavioral, and neurobiological account of a prefrontal reliability-based arbitration system

Hierarchical models of pain: Inference, information-seeking, and adaptive control.

Computational models of pain consider how the brain processes nociceptive information and allow mapping neural circuits and networks to cognition and behaviour. To date, they have generally have assumed two largely independent processes: perceptual inference, typically modelled as an approximate Bayesian process, and action control, typically modelled as a reinforcement learning process. However, inference and control are intertwined in complex ways, challenging the clarity of this distinction. Here, we consider how they may comprise a parallel hierarchical architecture that combines inference, information-seeking, and adaptive value-based control. This sheds light on the complex neural architecture of the pain system, and takes us closer to understanding from where pain 'arises' in the brain

Reward Enhances Pain Discrimination in Humans

The notion that reward inhibits pain is a well-supported observation in both humans and animals, allowing suppression of pain reflexes to acquired rewarding stimuli. However, a blanket inhibition of pain by reward would also impair pain discrimination. In contrast, early counterconditioning experiments implied that reward might actually spare pain discrimination. To test this hypothesis, we investigated whether discriminative performance was enhanced or inhibited by reward. We found in adult human volunteers (N = 25) that pain-based discriminative ability is actually enhanced by reward, especially when reward is directly contingent on discriminative performance. Drift-diffusion modeling shows that this relates to an augmentation of the underlying sensory signal strength and is not merely an effect of decision bias. This enhancement of sensory-discriminative pain-information processing suggests that whereas reward can promote reward-acquiring behavior by inhibition of pain in some circumstances, it can also facilitate important discriminative information of the sensory input when necessary

Jet Observables of Parton Energy Loss in High-Energy Nuclear Collisions

While strong attenuation of single particle production and particle correlations has provided convincing evidence for large parton energy loss in the QGP, its application to jet tomography has inherent limitations due to the inclusive nature of the measurements. Generalization of this suppression to full jet observables leads to an unbiased, more differential and thus powerful approach to determining the characteristics of the hot QCD medium created in high-energy nuclear collisions. In this article we report on recent theoretical progress in calculating jet shapes and the related jet cross sections in the presence of QGP-induced parton energy loss. (i) A theoretical model of intra-jet energy flow in heavy-ion collisions is discussed. (ii) Realistic numerical simulations demonstrate the nuclear modification factor $R_{AA}(p_T)$ evolves continuously with the jet cone size $R^{\max}$ or the acceptance cut $\omega_{\min}$ - a novel feature of jet quenching. The anticipated broadening of jets is subtle and most readily manifested in the periphery of the cone for smaller cone radii.Comment: Proceedings for Quark Matter 2009, updated version with minor correction

Value generalization in human avoidance learning.

Generalization during aversive decision-making allows us to avoid a broad range of potential threats following experience with a limited set of exemplars. However, over-generalization, resulting in excessive and inappropriate avoidance, has been implicated in a variety of psychological disorders. Here, we use reinforcement learning modelling to dissect out different contributions to the generalization of instrumental avoidance in two groups of human volunteers (N = 26, N = 482). We found that generalization of avoidance could be parsed into perceptual and value-based processes, and further, that value-based generalization could be subdivided into that relating to aversive and neutral feedback - with corresponding circuits including primary sensory cortex, anterior insula, amygdala and ventromedial prefrontal cortex. Further, generalization from aversive, but not neutral, feedback was associated with self-reported anxiety and intrusive thoughts. These results reveal a set of distinct mechanisms that mediate generalization in avoidance learning, and show how specific individual differences within them can yield anxiety.Wellcome, Arthritis Research U

Insula and Striatum Mediate the Default Bias

Humans are creatures of routine and habit. When faced with situations in which a default option is available, people show a consistent tendency to stick with the default. Why this occurs is unclear. To elucidate its neural basis, we used a novel gambling task in conjunction with functional magnetic resonance imaging. Behavioral results revealed that participants were more likely to choose the default card and felt enhanced emotional responses to outcomes after making the decision to switch. We show that increased tendency to switch away from the default during the decision phase was associated with decreased activity in the anterior insula; activation in this same area in reaction to “switching away from the default and losing” was positively related with experienced frustration. In contrast, decisions to choose the default engaged the ventral striatum, the same reward area as seen in winning. Our findings highlight aversive processes in the insula as underlying the default bias and suggest that choosing the default may be rewarding in itself