Cooperation in the iterated prisoner's dilemma is learned by operant conditioning mechanisms

The prisoner's dilemma (PD) is the leading metaphor for the evolution of cooperative behavior in populations of selfish agents. Although cooperation in the iterated prisoner's dilemma (IPD) has been studied for over twenty years, most of this research has been focused on strategies that involve nonlearned behavior. Another approach is to suppose that players' selection of the preferred reply might he enforced in the same way as feeding animals track the best way to feed in changing nonstationary environments. Learning mechanisms such as operant conditioning enable animals to acquire relevant characteristics of their environment in order to get reinforcements and to avoid punishments. In this study, the role of operant conditioning in the learning of cooperation was evaluated in the PD. We found that operant mechanisms allow the learning of IPD play against other strategies. When random moves are allowed in the game, the operant learning model showed low sensitivity. On the basis of this evidence, it is suggested that operant learning might be involved in reciprocal altruism.Fil: Gutnisky, D. A.. Universidad de Buenos Aires. Facultad de Ingenieria. Instituto de Ingeniería Biomédica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingenieria. Instituto de Ingeniería Biomédica; Argentin

Distinct fos-expressing neuronal ensembles in the ventromedial prefrontal cortex mediate food reward and extinction memories

In operant learning, initial reward-associated memories are thought to be distinct from subsequent extinction-associated memories. Memories formed during operant learning are thought to be stored in “neuronal ensembles.” Thus, we hypothesize that different neuronal ensembles encode reward- and extinction-associated memories. Here, we examined prefrontal cortex neuronal ensembles involved in the recall of reward and extinction memories of food self-administration.Wefirst trained rats to lever press for palatable food pellets for 7 d (1 h/d) and then exposed them to 0, 2, or 7 daily extinction sessions in which lever presses were not reinforced. Twenty-four hours after the last training or extinction session, we exposed the rats to either a short 15 min extinction test session or left them in their homecage (a control condition). We found maximal Fos (a neuronal activity marker) immunoreactivity in the ventral medial prefrontal cortex of rats that previously received 2 extinction sessions, suggesting that neuronal ensembles in this area encode extinction memories. We then used the Daun02 inactivation procedure to selectively disrupt ventral medial prefrontal cortex neuronal ensembles that were activated during the 15 min extinction session following 0 (no extinction) or 2 prior extinction sessions to determine the effects of inactivating the putative food reward and extinction ensembles, respectively, on subsequent nonreinforced food seeking 2 d later. Inactivation of the food reward ensembles decreased food seeking, whereas inactivation of the extinction ensembles increased food seeking. Our results indicate that distinct neuronal ensembles encoding operant reward and extinction memories intermingle within the same cortical area

Applying psychological theory to evidence-based clinical practice : identifying factors predictive of managing upper respiratory tract infections without antibiotics

Background: Psychological models can be used to understand and predict behaviour in a wide range of settings. However, they have not been consistently applied to health professional behaviours, and the contribution of differing theories is not clear. The aim of this study was to explore the usefulness of a range of psychological theories to predict health professional behaviour relating to management of upper respiratory tract infections (URTIs) without antibiotics. Methods: Psychological measures were collected by postal questionnaire survey from a random sample of general practitioners (GPs) in Scotland. The outcome measures were clinical behaviour (using antibiotic prescription rates as a proxy indicator), behavioural simulation (scenario-based decisions to managing URTI with or without antibiotics) and behavioural intention (general intention to managing URTI without antibiotics). Explanatory variables were the constructs within the following theories: Theory of Planned Behaviour (TPB), Social Cognitive Theory (SCT), Common Sense Self-Regulation Model (CS-SRM), Operant Learning Theory (OLT), Implementation Intention (II), Stage Model (SM), and knowledge (a non-theoretical construct). For each outcome measure, multiple regression analysis was used to examine the predictive value of each theoretical model individually. Following this 'theory level' analysis, a 'cross theory' analysis was conducted to investigate the combined predictive value of all significant individual constructs across theories. Results: All theories were tested, but only significant results are presented. When predicting behaviour, at the theory level, OLT explained 6% of the variance and, in a cross theory analysis, OLT 'evidence of habitual behaviour' also explained 6%. When predicting behavioural simulation, at the theory level, the proportion of variance explained was: TPB, 31%; SCT, 26%; II, 6%; OLT, 24%. GPs who reported having already decided to change their management to try to avoid the use of antibiotics made significantly fewer scenario-based decisions to prescribe. In the cross theory analysis, perceived behavioural control (TPB), evidence of habitual behaviour (OLT), CS-SRM cause (chance/bad luck), and intention entered the equation, together explaining 36% of the variance. When predicting intention, at the theory level, the proportion of variance explained was: TPB, 30%; SCT, 29%; CS-SRM 27%; OLT, 43%. GPs who reported that they had already decided to change their management to try to avoid the use of antibiotics had a significantly higher intention to manage URTIs without prescribing antibiotics. In the cross theory analysis, OLT evidence of habitual behaviour, TPB attitudes, risk perception, CS-SRM control by doctor, TPB perceived behavioural control and CS-SRM control by treatment entered the equation, together explaining 49% of the variance in intention. Cnclusion: The study provides evidence that psychological models can be useful in understanding and predicting clinical behaviour. Taking a theory-based approach enables the reation of a replicable methodology for identifying factors that predict clinical behaviour. Hwever, a number of conceptual and methodological challenges remain

Implicit operant learning of pain sensitization and habituation in healthy participants and fibromyalgia patients

The important role of operant learning in chronic pain is widely recognized. However, the precise mechanisms mediating between nociceptive processing, operant consequences and altered pain perception remain being unclear. The general aim of the three studies contained in this thesis was to clarify the latter. For this purpose, experimental operant learning tasks were employed that were independent of subjective pain report, in order to avoid the risk of solely changing response crite-ria. Further, intrinsic reinforcement—within the nociceptive system by reductions in nociceptive input—was applied. In contrast to extrinsic reinforcement—external to the nociceptive system e.g. by monetary reinforcement—intrinsic reinforcement directly affects pain perception. In study 1, a continuous operant conditioning procedure for enhancing pain sensitivity with different magnitudes of reinforcement and different schedules of reinforcement was implemented in healthy participants. The results indicated a dose-dependency of the operant learning. Further, enhanced pain sensitivity was the result of underlying learning rather than immediate (unconditioned) effects of reinforcement (pain relief). Together with a previous study, the independence of the operant learning mechanism from the experimental procedure was demonstrated. Thus, operant learning of pain sensitivity was demonstrated to be a valid and robust mechanism. In order to demonstrate operant learning to be implicit—i.e. learning without awareness—study 2 employed a discrete-trial operant learning procedure in healthy participants similar to a previously implemented operant learning procedure. Awareness was tested with a behavioral task (prediction of reinforcement) and a standardized interview, addressing different levels of processing. The results demonstrated that operant learning of altered pain sensitivity was implicit; neither verbalization of the operant contingencies (the relationships between behavior and stimuli associated with reinforcement) nor behavioral discrimination was necessary for successful operant learning. Study 3 repeated the same paradigm as study 2 in chronic pain patients with fibromyalgia with and without comorbid irritable bowel syndrome for testing the vulnerability of these patients to operant learning of altered pain sensitivity. In contrast to a hypothesized heightened vulnerability, operant learning was impaired in these patients compared to healthy participants. Moreover, fibromyalgia patients with and without irritable bowel syndrome respond differentially to the operant conditioning: While fibromyalgia patients with irritable bowel syndrome showed no signs of operant learning, fibromyalgia patients without irritable bowel syndrome displayed enhanced perceptual sensitization, but this enhancement was paradoxically more pronounced in the habituation learning condition than in the sensitization learning condition. Thus, parameters of operant learning of altered pain sensitivity differentiated between healthy participants and chronic pain patients as well as between different groups of chronic pain patients. In addition, a dissociation of physical stimulus intensities and subjective pain reports was observed, indicating that overt reporting is not the adequate representation level of pain perception relevant for learning altered pain sensitivity. Taken together, these studies demonstrated (1) operant learning to a be powerful, pathogenetically relevant mechanism, producing gross changes in pain sensitivity without the persons’ knowledge and (2) the necessity of behavioral discrimination paradigms that do not depend on subjective measures of pain expe-rience in the first place. The proposed implicit operant learning mechanism of altered pain sensitivity with intrinsic reinforcement provides an explanation for the gradual development of hypersensitivity in pain that is becoming chronic. These results have implications for a wide range of applications ranging from diagnostic procedures to therapeutical interventions in chronic pain

Pain rewarded: hyperalgesic and allodynic effect of operant conditioning in healthy humans : protocol for a systematic review and meta-analysis

Background: "pain rewarded" is a hypothesis wherein acute pain sufferers are exposed to reinforcers and punishers from their environment that shape their behaviour, i.e. pain responses. Such a point of view has been taken for granted by many clinicians and researchers although existing evidence has not yet been systematically summarized. This planned systematic review and meta-analysis is aiming to summarize the research findings on pain modulation (hyperalgesic effect) and pain elicitation (allodynic effect) resulting from operant conditioning procedures in healthy humans. Methods: the systematic review will be performed by searching for articles indexed in PubMed database, Cochrane Register of Controlled Trials (Central), Web of Science™, ScienceDirect, Ebsco database, PsycInfo, Medline, PsycArticles and cinahl. Studies will be included if they investigate healthy humans, exposed to modulation or elicitation of a pain experience induced by operant conditioning. Studies will be screened for eligibility and risk of bias by two independent assessors. Narrative and meta-analytical syntheses are planned. Discussion: data will be pooled and analyzed qualitatively and quantitatively (if possible) in order to advance the understanding of pain mechanisms, especially the development of chronic pain. This systematic review will guide the planning of future experiments and research by summarizing important technical details of conditioning procedures in healthy humans

Learning obstacle avoidance with an operant behavioral model

Artificial intelligence researchers have been attracted by the idea of having robots learn how to accomplish a task, rather than being told explicitly. Reinforcement learning has been proposed as an appealing framework to be used in controlling mobile agents. Robot learning research, as well as research in biological systems, face many similar problems in order to display high flexibility in performing a variety of tasks. In this work, the controlling of a vehicle in an avoidance task by a previously developed operant learning model (a form of animal learning) is studied. An environment in which a mobile robot with proximity sensors has to minimize the punishment for colliding against obstacles is simulated. The results were compared with the Q-Learning algorithm, and the proposed model had better performance. In this way a new artificial intelligence agent inspired by neurobiology, psychology, and ethology research is proposed.Fil: Gutnisky, D. A.. Universidad de Buenos Aires. Facultad de Ingeniería.Instituto de Ingeniería Biomédica; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingeniería.Instituto de Ingeniería Biomédica; Argentin

Doubling your payoff: winning pain relief engages endogenous pain inhibition

When in pain, pain relief is much sought after, particularly for individuals with chronic pain. In analogy to augmentation of the hedonic experience (“liking”) of a reward by the motivation to obtain a reward (“wanting”), the seeking of pain relief in a motivated state might increase the experience of pain relief when obtained. We tested this hypothesis in a psychophysical experiment in healthy human subjects, by assessing potential pain-inhibitory effects of pain relief “won” in a wheel of fortune game compared with pain relief without winning, exploiting the fact that the mere chance of winning induces a motivated state. The results show pain-inhibitory effects of pain relief obtained by winning in behaviorally assessed pain perception and ratings of pain intensity. Further, the higher participants scored on the personality trait novelty seeking, the more pain inhibition was induced. These results provide evidence that pain relief, when obtained in a motivated state, engages endogenous pain-inhibitory systems beyond the pain reduction that underlies the relief in the first place. Consequently, such pain relief might be used to improve behavioral pain therapy, inducing a positive, perhaps self-amplifying feedback loop of reduced pain and improved functionality

A break from pain! Interruption management in the context of pain

Activity interruptions, namely temporary suspensions of an ongoing task with the intention to resume it later, are common in pain. First, pain is a threat signal that urges us to interrupt ongoing activities in order to manage the pain and its cause. Second, activity interruptions are used in chronic pain management. However, activity interruptions by pain may carry costs for activity performance. These costs have recently started to be systematically investigated. We review the evidence on the consequences of activity interruptions by pain for the performance of the interrupted activity. Further, inspired by literature on interruptions from other research fields, we suggest ways to improve interruption management in the field of pain, and provide a future research agenda

Somatisation and functional impairment in adolescents: longitudinal link with mothers' reactions

Adolescents' somatisation (i.e., the psychological tendency to experience and report multiple physical complaints for which no definite medical cause can be found; SOM) and functional impairment (i.e., all bothersome aftermath of somatisation; FI) were studied in relation to mothers' protection, encouraging/monitoring, and minimisation of physical functional complaints. Besides main effects, interaction effects with other child and parenting characteristics were examined. A total of 990 adolescents and their mothers filled out questionnaires when the adolescents were respectively 12-13 (T1) and 13-14 (T2) years old. At T1, there was a significant relation between mothers' higher amounts of minimisation and adolescents' higher levels of SOM. Further, the link between mothers' higher levels of T1 minimisation and adolescents' higher amounts of T1 FI was significant, but not for adolescents with high levels of depressive mood. Longitudinal analyses revealed that mothers' reactions did not significantly predict adolescents' SOM/FI, nor did adolescents' SOM/FI significantly predict mothers' reactions. Practical implications are discussed

Evidence-based selection of theories for designing behaviour change interventions : using methods based on theoretical construct domains to understand clinicians' blood transfusion behaviour

Peer reviewedPostprin