Cooperation in the iterated prisoner's dilemma is learned by operant conditioning mechanisms

The prisoner's dilemma (PD) is the leading metaphor for the evolution of cooperative behavior in populations of selfish agents. Although cooperation in the iterated prisoner's dilemma (IPD) has been studied for over twenty years, most of this research has been focused on strategies that involve nonlearned behavior. Another approach is to suppose that players' selection of the preferred reply might he enforced in the same way as feeding animals track the best way to feed in changing nonstationary environments. Learning mechanisms such as operant conditioning enable animals to acquire relevant characteristics of their environment in order to get reinforcements and to avoid punishments. In this study, the role of operant conditioning in the learning of cooperation was evaluated in the PD. We found that operant mechanisms allow the learning of IPD play against other strategies. When random moves are allowed in the game, the operant learning model showed low sensitivity. On the basis of this evidence, it is suggested that operant learning might be involved in reciprocal altruism.Fil: Gutnisky, D. A.. Universidad de Buenos Aires. Facultad de Ingenieria. Instituto de Ingeniería Biomédica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingenieria. Instituto de Ingeniería Biomédica; Argentin

Learning obstacle avoidance with an operant behavioral model

Artificial intelligence researchers have been attracted by the idea of having robots learn how to accomplish a task, rather than being told explicitly. Reinforcement learning has been proposed as an appealing framework to be used in controlling mobile agents. Robot learning research, as well as research in biological systems, face many similar problems in order to display high flexibility in performing a variety of tasks. In this work, the controlling of a vehicle in an avoidance task by a previously developed operant learning model (a form of animal learning) is studied. An environment in which a mobile robot with proximity sensors has to minimize the punishment for colliding against obstacles is simulated. The results were compared with the Q-Learning algorithm, and the proposed model had better performance. In this way a new artificial intelligence agent inspired by neurobiology, psychology, and ethology research is proposed.Fil: Gutnisky, D. A.. Universidad de Buenos Aires. Facultad de Ingeniería.Instituto de Ingeniería Biomédica; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingeniería.Instituto de Ingeniería Biomédica; Argentin

The N2pc Is Increased by Perceptual Learning but Is Unnecessary for the Transfer of Learning

Background: Practice improves human performance in many psychophysical paradigms. This kind of improvement is thought to be the evidence of human brain plasticity. However, the changes that occur in the brain are not fully understood. Methodology/Principal Findings: The N2pc component has previously been associated with visuo-spatial attention. In this study, we used event-related potentials (ERPs) to investigate whether the N2pc component changed during long-term visual perceptual learning. Thirteen subjects completed several days of training in an orientation discrimination task, and were given a final test 30 days later. The results showed that behavioral thresholds significantly decreased across training sessions, and this decrement was also present in the untrained visual field. ERPs showed training significantly increased the N2pc amplitude, and this effect could be maintained for up to 30 days. However, the increase in N2pc was specific to the trained visual field. Conclusion/Significance: Training caused spatial attention to be increasingly focused on the target positions. However, this process was not transferrable from the trained to the untrained visual field, which suggests that the increase in N2pc ma

Functional Clustering Drives Encoding Improvement in a Developing Brain Network during Awake Visual Learning

Sensory experience drives dramatic structural and functional plasticity in developing neurons. However, for single-neuron plasticity to optimally improve whole-network encoding of sensory information, changes must be coordinated between neurons to ensure a full range of stimuli is efficiently represented. Using two-photon calcium imaging to monitor evoked activity in over 100 neurons simultaneously, we investigate network-level changes in the developing Xenopus laevis tectum during visual training with motion stimuli. Training causes stimulus-specific changes in neuronal responses and interactions, resulting in improved population encoding. This plasticity is spatially structured, increasing tuning curve similarity and interactions among nearby neurons, and decreasing interactions among distant neurons. Training does not improve encoding by single clusters of similarly responding neurons, but improves encoding across clusters, indicating coordinated plasticity across the network. NMDA receptor blockade prevents coordinated plasticity, reduces clustering, and abolishes whole-network encoding improvement. We conclude that NMDA receptors support experience-dependent network self-organization, allowing efficient population coding of a diverse range of stimuli.Canadian Institutes of Health Researc

Balanced Synaptic Input Shapes the Correlation between Neural Spike Trains

Stimulus properties, attention, and behavioral context influence correlations between the spike times produced by a pair of neurons. However, the biophysical mechanisms that modulate these correlations are poorly understood. With a combined theoretical and experimental approach, we show that the rate of balanced excitatory and inhibitory synaptic input modulates the magnitude and timescale of pairwise spike train correlation. High rate synaptic inputs promote spike time synchrony rather than long timescale spike rate correlations, while low rate synaptic inputs produce opposite results. This correlation shaping is due to a combination of enhanced high frequency input transfer and reduced firing rate gain in the high input rate state compared to the low state. Our study extends neural modulation from single neuron responses to population activity, a necessary step in understanding how the dynamics and processing of neural activity change across distinct brain states

Thalamic Activation Modulates the Responses of Neurons in Rat Primary Auditory Cortex: An In Vivo Intracellular Recording Study

Auditory cortical plasticity can be induced through various approaches. The medial geniculate body (MGB) of the auditory thalamus gates the ascending auditory inputs to the cortex. The thalamocortical system has been proposed to play a critical role in the responses of the auditory cortex (AC). In the present study, we investigated the cellular mechanism of the cortical activity, adopting an in vivo intracellular recording technique, recording from the primary auditory cortex (AI) while presenting an acoustic stimulus to the rat and electrically stimulating its MGB. We found that low-frequency stimuli enhanced the amplitudes of sound-evoked excitatory postsynaptic potentials (EPSPs) in AI neurons, whereas high-frequency stimuli depressed these auditory responses. The degree of this modulation depended on the intensities of the train stimuli as well as the intervals between the electrical stimulations and their paired sound stimulations. These findings may have implications regarding the basic mechanisms of MGB activation of auditory cortical plasticity and cortical signal processing

Adaptation-Dependent Synchronous Activity Contributes to Receptive Field Size Change of Bullfrog Retinal Ganglion Cell

Nearby retinal ganglion cells of similar functional subtype have a tendency to discharge spikes in synchrony. The synchronized activity is involved in encoding some aspects of visual input. On the other hand, neurons always continuously adjust their activities in adaptation to some features of visual stimulation, including mean ambient light, contrast level, etc. Previous studies on adaptation were primarily focused on single neuronal activity, however, it is also intriguing to investigate the adaptation process in population neuronal activities. In the present study, by using multi-electrode recording system, we simultaneously recorded spike discharges from a group of dimming detectors (OFF-sustained type ganglion cells) in bullfrog retina. The changes in receptive field properties and synchronization strength during contrast adaptation were analyzed. It was found that, when perfused using normal Ringer's solution, single neuronal receptive field size was reduced during contrast adaptation, which was accompanied by weakening in synchronization strength between adjacent neurons' activities. When dopamine (1 µM) was applied, the adaptation-related receptive field area shrinkage and synchronization weakening were both eliminated. The activation of D1 receptor was involved in the adaptation-related modulation of synchronization and receptive field. Our results thus suggest that the size of single neuron's receptive field is positively related to the strength of its synchronized activity with its neighboring neurons, and the dopaminergic pathway is responsible for the modulation of receptive field property and synchronous activity of the ganglion cells during the adaptation process

Mechanisms underlying a thalamocortical transformation during active tactile sensation

During active somatosensation, neural signals expected from movement of the sensors are suppressed in the cortex, whereas information related to touch is enhanced. This tactile suppression underlies low-noise encoding of relevant tactile features and the brain’s ability to make fine tactile discriminations. Layer (L) 4 excitatory neurons in the barrel cortex, the major target of the somatosensory thalamus (VPM), respond to touch, but have low spike rates and low sensitivity to the movement of whiskers. Most neurons in VPM respond to touch and also show an increase in spike rate with whisker movement. Therefore, signals related to self-movement are suppressed in L4. Fast-spiking (FS) interneurons in L4 show similar dynamics to VPM neurons. Stimulation of halorhodopsin in FS interneurons causes a reduction in FS neuron activity and an increase in L4 excitatory neuron activity. This decrease of activity of L4 FS neurons contradicts the "paradoxical effect" predicted in networks stabilized by inhibition and in strongly-coupled networks. To explain these observations, we constructed a model of the L4 circuit, with connectivity constrained by in vitro measurements. The model explores the various synaptic conductance strengths for which L4 FS neurons actively suppress baseline and movement-related activity in layer 4 excitatory neurons. Feedforward inhibition, in concert with recurrent intracortical circuitry, produces tactile suppression. Synaptic delays in feedforward inhibition allow transmission of temporally brief volleys of activity associated with touch. Our model provides a mechanistic explanation of a behavior-related computation implemented by the thalamocortical circuit

Congenital hypothyroidism

Congenital hypothyroidism (CH) occurs in approximately 1:2,000 to 1:4,000 newborns. The clinical manifestations are often subtle or not present at birth. This likely is due to trans-placental passage of some maternal thyroid hormone, while many infants have some thyroid production of their own. Common symptoms include decreased activity and increased sleep, feeding difficulty, constipation, and prolonged jaundice. On examination, common signs include myxedematous facies, large fontanels, macroglossia, a distended abdomen with umbilical hernia, and hypotonia. CH is classified into permanent and transient forms, which in turn can be divided into primary, secondary, or peripheral etiologies. Thyroid dysgenesis accounts for 85% of permanent, primary CH, while inborn errors of thyroid hormone biosynthesis (dyshormonogeneses) account for 10-15% of cases. Secondary or central CH may occur with isolated TSH deficiency, but more commonly it is associated with congenital hypopitiutarism. Transient CH most commonly occurs in preterm infants born in areas of endemic iodine deficiency. In countries with newborn screening programs in place, infants with CH are diagnosed after detection by screening tests. The diagnosis should be confirmed by finding an elevated serum TSH and low T4 or free T4 level. Other diagnostic tests, such as thyroid radionuclide uptake and scan, thyroid sonography, or serum thyroglobulin determination may help pinpoint the underlying etiology, although treatment may be started without these tests. Levothyroxine is the treatment of choice; the recommended starting dose is 10 to 15 mcg/kg/day. The immediate goals of treatment are to rapidly raise the serum T4 above 130 nmol/L (10 ug/dL) and normalize serum TSH levels. Frequent laboratory monitoring in infancy is essential to ensure optimal neurocognitive outcome. Serum TSH and free T4 should be measured every 1-2 months in the first 6 months of life and every 3-4 months thereafter. In general, the prognosis of infants detected by screening and started on treatment early is excellent, with IQs similar to sibling or classmate controls. Studies show that a lower neurocognitive outcome may occur in those infants started at a later age (> 30 days of age), on lower l-thyroxine doses than currently recommended, and in those infants with more severe hypothyroidism