A Computational Theory for the Learning of Equivalence Relations

Equivalence relations (ERs) are logical entities that emerge concurrently with the development of language capabilities. In this work we propose a computational model that learns to build ERs by learning simple conditional rules. The model includes visual areas, dopaminergic, and noradrenergic structures as well as prefrontal and motor areas, each of them modeled as a group of continuous valued units that simulate clusters of real neurons. In the model, lateral interaction between neurons of visual structures and top-down modulation of prefrontal/premotor structures over the activity of neurons in visual structures are necessary conditions for learning the paradigm. In terms of the number of neurons and their interaction, we show that a minimal structural complexity is required for learning ERs among conditioned stimuli. Paradoxically, the emergence of the ER drives a reduction in the number of neurons needed to maintain those previously specific stimulus–response learned rules, allowing an efficient use of neuronal resources

Cooperation in the iterated prisoner's dilemma is learned by operant conditioning mechanisms

The prisoner's dilemma (PD) is the leading metaphor for the evolution of cooperative behavior in populations of selfish agents. Although cooperation in the iterated prisoner's dilemma (IPD) has been studied for over twenty years, most of this research has been focused on strategies that involve nonlearned behavior. Another approach is to suppose that players' selection of the preferred reply might he enforced in the same way as feeding animals track the best way to feed in changing nonstationary environments. Learning mechanisms such as operant conditioning enable animals to acquire relevant characteristics of their environment in order to get reinforcements and to avoid punishments. In this study, the role of operant conditioning in the learning of cooperation was evaluated in the PD. We found that operant mechanisms allow the learning of IPD play against other strategies. When random moves are allowed in the game, the operant learning model showed low sensitivity. On the basis of this evidence, it is suggested that operant learning might be involved in reciprocal altruism.Fil: Gutnisky, D. A.. Universidad de Buenos Aires. Facultad de Ingenieria. Instituto de Ingeniería Biomédica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingenieria. Instituto de Ingeniería Biomédica; Argentin

Learning obstacle avoidance with an operant behavioral model

Artificial intelligence researchers have been attracted by the idea of having robots learn how to accomplish a task, rather than being told explicitly. Reinforcement learning has been proposed as an appealing framework to be used in controlling mobile agents. Robot learning research, as well as research in biological systems, face many similar problems in order to display high flexibility in performing a variety of tasks. In this work, the controlling of a vehicle in an avoidance task by a previously developed operant learning model (a form of animal learning) is studied. An environment in which a mobile robot with proximity sensors has to minimize the punishment for colliding against obstacles is simulated. The results were compared with the Q-Learning algorithm, and the proposed model had better performance. In this way a new artificial intelligence agent inspired by neurobiology, psychology, and ethology research is proposed.Fil: Gutnisky, D. A.. Universidad de Buenos Aires. Facultad de Ingeniería.Instituto de Ingeniería Biomédica; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingeniería.Instituto de Ingeniería Biomédica; Argentin

Probing the structure–function relationship with neural networks constructed by solving a system of linear equations

Neural network models are an invaluable tool to understand brain function since they allow us to connect the cellular and circuit levels with behaviour. Neural networks usually comprise a huge number of parameters, which must be chosen carefully such that networks reproduce anatomical, behavioural, and neurophysiological data. These parameters are usually fitted with off-the-shelf optimization algorithms that iteratively change network parameters and simulate the network to evaluate its performance and improve fitting. Here we propose to invert the fitting process by proceeding from the network dynamics towards network parameters. Firing state transitions are chosen according to the transition graph associated with the solution of a task. Then, a system of linear equations is constructed from the network firing states and membrane potentials, in a way that guarantees the consistency of the system. This allows us to uncouple the dynamical features of the model, like its neurons firing rate and correlation, from the structural features, and the task-solving algorithm implemented by the network. We employed our method to probe the structure–function relationship in a sequence memory task. The networks obtained showed connectivity and firing statistics that recapitulated experimental observations. We argue that the proposed method is a complementary and needed alternative to the way neural networks are constructed to model brain function.Fil: Mininni, Camilo Juan. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingeniería. Instituto de Ingeniería Biomédica.; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingeniería. Instituto de Ingeniería Biomédica.; Argentin

Control nervioso del sistema circulatorio : estudio experimental y su formalización utilizando redes neuronales

Las variables cardiovasculares están controladas pormecanismos humorales, nerviosos, y de autorregulación. Losmismos, interactúan durante toda la vida del individuo, peroavanzada lo ontogenia, el sistema nervioso cumple una funciónpreponderante en la regulación a largo plazo, de modo tal, quesi se anulan ciertas estructuras nerviosas, no puedenmantenerse los valores medios. En el feto y en el neonato, losmecanismosde regulación nerviosa de la circulación no estántotalmente desarrollados; por lo tanto, para la elaboración deun modelo de control, se consideró necesario tener en cuentala ontogenia del sistema. En este trabajo se analizan ciertos aspectos de la funcióndel sistema nervioso en el control de la circulación. Sepropone que el núcleo del tracto solitario (NTS) tiene comopropiedad emergente la de ser un comparador sobre el cualactúa la referencia. De esta manerael sistema de controltendría, un lazo realimentado por barorreceptores,quimiorreceptores y receptores cardiopulmonares, mientras queciertas estructuras rostrales al NTS darían la referencia. Poresta razón se realizaron experimentos en la rata blancaanestesiada (como modelo animal adecuado), para analizar lasrespuestas a las interacciones entre la estimulación eléctricade distintas estructuras rostrales al NTS y perturbacionesdentro del lazo realimentado. Se utilizaron comoperturbaciones, la estimulación eléctrica de la formaciónreticulada ventrolateral (FRVL) y la oclusión carotidea (OC)del lado proximal al corazón. Fueron medidas en formacontinua: la presión arterial, la frecuencia cardiaca, larespiración, el electrocardiograma, y el electrocorticograma. Se observó que la estimulación eléctrica del área septallateral y el hipotálamo lateral puede bloquear o potenciar lasrespuestas a la estimulación eléctrica de la FRVL y bloquearlas respuestas a la (OC). Además, se vio que la estimulacióndel hipocampodorsal y el fórnix potencian las respuestas a laestimulación de la FRVL, mientras que la del hipocampo ventrallas bloquea. Tambien, la estimulación de estas estructurasproduce una potenciación tónica de la respuestas a laestimulación de la FRVL. No se observaron interacciones entrelas respuestas a la estimulación del área septal medial y la FRVL, ni tampoco entre la FRVL y la OC. Durante lasinteracciones fásicas, no se observaron cambios en lascatecolaminas circulantes (adrenalina, noradrenalina ydopamina). Lo cual indica que las mismas no están involucradasen estos fenómenos. Cuandose obtuvieron respuestas a laestimulación de los núcleos rostrales al NTS, las respuestas alas perturbaciones en estructuras dentro del lazo realimentadono presentaron cambios. Estos resultados confirman que el NTStiene la propiedad emergente de ser un comparador. Mostrandoademás, que los restantes núcleos rostrales estudiados, puedenmodular las respuestas del lazo realimentado. Para analizarlas propiedades mencionadas, se propone un modelo de redesneuronales que permite estudiar el proceso de la ontogenia delcontrol, donde queda definida la estructura del comparador ycómo actúa la referencia. Se postula que el sistema nerviosose adapta a través de procesos plásticos, que son modeladoscomo un aprendizaje en base a la hipótesis de Hebb. El sistemaaprende las salidas simpáticas a partir de la información delos quimiorreceptores (que ya funcionan en el feto y en elrecién nacido). La red neuronal está formada por un número denodos (que simulan neuronas) igual al número de tejidos. Cadauno tiene dos entradas, una proveniente de losquimiorreceptores, y la otra de núcleos rostrales al NTS. Lasalida envía eferentes para el control del flujo en los vasosde los diferentes tejidos. El sistema conyerge hacia unestado, donde en condiciones de metabolismo normal, lostejidos están irrigados con presiones parciales de oxigeno ydióxido de carbono normales, y los quimiorreceptores nodescargan. Al variar las condiciones metabólicas, se modificala realimentación de los receptores, cambiando el controlnervioso y la participación de los mecanismos humorales y deautorregulación. Para ciertas conductas, los núcleos rostralesal NTS podrian modificar la referencia, (compuesta por unconjunto de entradas) determinando así, el flujo sanguíneo enlos distintos tejidos. Además, podrian modular las variablescardiovasculares para modificar el flujo sanguíneo según lasnecesidades.The cardiovascular variables are controlled by humoral,nervous and autoregulatory mechanisms. These phenomena,interact during the whole life of the individual, but once theontogeny goes forward, the nervous system has a prevalent roleduring long-term regulation, in such a way that, if somenervous structures are blocked-out, it becomes impossible tomaintain the mean value. In the fetus and the newborn, themechanisms of nervous regulation of the circulation are notcompletely developed; therefore, in order to elaborate of amodelof control, it is considered necessary to take intoaccount the system ontogeny. Some aspects of the nervous control of circulation areanalyzed in this work. It is proposed that the nucleus of thetractus solitarius (NTS) has the emergent property to be acomparator, over which the reference acts. In this way, thesystem of control would have in its feedback loop baro, chemoand cardiopulmonary receptors. Some structures rostral to the NTS would give the reference. Experiments were carried out inanesthetized white rat, to analyze the responses to theinteraction betweenthe electrical stimulation of differentstructures rostral to the NTSand perturbations within thefeedback loop. The electrical stimulation of the ventrolateralreticular formation (VLRF)and carotid occlusion (CO) proximalto the heart were used as a perturbation. Blood pressure,heart rate, respiration, electrocardiogram andelectrocorticogram, were recorded continuously. It wasobserved that the electrical stimulation of the lateral septalarea and the lateral hypothalamus area, blocked or potentiatedresponses of VLRF electrical stimulation, and blocked theresponses to CO. Besides, it was observed that the dorsalhippocampus and the fornix stimulation potentiated responsesof VLRF stimulation, while the ventral hippocampus blocked it. Also a tonic potentiation of the responses of VLRF stimulationwas produced by the stimulation of these structures. Neitherthe interaction between the responses to the electricalstimulation of the medial septal area and VLRFstimulation,nor the interaction between VLRFand COwere observed. Duringthe phasic interaction, no changes in circulatorycatecholamine were noted, suggesting that they were notinvolved in the phenomenon. When responses to electricalstimulation of rostral nuclei to the NTS were obtained,responses to perturbations of structures inside the feedbackloop did not present changes. The idea that the NTS behaves asan emergent comparator was then confirmed. Moreover, theseresults shows, that the considered rostral nuclei, couldmodulate the responses of the feedback loop. To analyze thementioned property, a neural network model is proposed,permitting the study of the ontogeny of the control, so as todefine the structure of the comparator and the way that thereference acts. It is postulated that the nervous systemadapted itself by plastic mechanisms, which are modulated as alearning process on the basis of the hebbian hypothesis. Thesystem learns the sympathetic output from the chemoreceptorinformation (which are functioning in the fetus and thenewborn). The neural network is composed by a number of nodes (simulating neurons) equal to the number of tissues undercontrol. Each one has two inputs, one from the chemoreceptorsand the other from the nuclei rostral to the NTS. The outputof each of these nodes sends efferents to control the flow ofthe vessels of the different tissues. The system converges toa state, where, in case of normal metabolism, the tissuesreceive a blood flow with a normal partial pressure of oxygenand carbon dioxide, and there are no discharges from thechemoreceptors. During metabolism drift, the receptor feedbackis modified, changing the nervous control as well as humoraland the autoregulatory mechanisms. In certain behaviors, thereference (composed by a set of inputs) could be modified bynuclei rostral to the NTS, determining in this way, the bloodflow in the different tissues. Besides, they could modulatethe cardiovascular variables to modify the blood flowaccording to particular needs.Fil: Zanutto, Bonifacio Silvano. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales; Argentina

Bang-Bang Control of Feeding: Role of Hypothalamic and Satiety Signals

Rats, people, and many other omnivores eat in meals rather than continuously. We show by experimental test that eating in meals is regulated by a simple bang-bang control system, an idea foreshadowed by Le Magnen and many others, shown by us to account for a wide range of behavioral data, but never explicitly tested or tied to neurophysiological facts. The hypothesis is simply that the tendency to eat rises with time at a rate determined by satiety signals. When these signals fall below a set point, eating begins, in on–off fashion. The delayed sequelae of eating increment the satiety signals, which eventually turn eating off. Thus, under free conditions, the organism eats in bouts separated by noneating activities. We report an experiment with rats to test novel predictions about meal patterns that are not explained by existing homeostatic approaches. Access to food was systematically but unpredictably interrupted just as the animal tried to start a new meal. A simple bang-bang model fits the resulting meal-pattern data well, and its elements can be identified with neurophysiological processes. Hypothalamic inputs can provide the set point for longer-term regulation carried out by a comparator in the hindbrain. Delayed gustatory and gastrointestinal aftereffects of eating act via the nucleus of the solitary tract and other hindbrain regions as neural feedback governing short-term regulation. In this way, the model forges real links between a functioning feedback mechanism, neuro–hormonal data, and both short-term (meals) and long-term (eating-rate regulation) behavioral data

Blood pressure long term regulation: A neural network model of the set point development

<p>Abstract</p> <p>Background</p> <p>The notion of the nucleus tractus solitarius (NTS) as a comparator evaluating the error signal between its rostral neural structures (RNS) and the cardiovascular receptor afferents into it has been recently presented. From this perspective, stress can cause hypertension via set point changes, so offering an answer to an old question. Even though the local blood flow to tissues is influenced by circulating vasoactive hormones and also by local factors, there is yet significant sympathetic control. It is well established that the state of maturation of sympathetic innervation of blood vessels at birth varies across animal species and it takes place mostly during the postnatal period. During ontogeny, chemoreceptors are functional; they discharge when the partial pressures of oxygen and carbon dioxide in the arterial blood are not normal.</p> <p>Methods</p> <p>The model is a simple biological plausible adaptative neural network to simulate the development of the sympathetic nervous control. It is hypothesized that during ontogeny, from the RNS afferents to the NTS, the optimal level of each sympathetic efferent discharge is learned through the chemoreceptors' feedback. Its mean discharge leads to normal oxygen and carbon dioxide levels in each tissue. Thus, the sympathetic efferent discharge sets at the optimal level if, despite maximal drift, the local blood flow is compensated for by autoregulation. Such optimal level produces minimum chemoreceptor output, which must be maintained by the nervous system. Since blood flow is controlled by arterial blood pressure, the long-term mean level is stabilized to regulate oxygen and carbon dioxide levels. After development, the cardiopulmonary reflexes play an important role in controlling efferent sympathetic nerve activity to the kidneys and modulating sodium and water excretion.</p> <p>Results</p> <p>Starting from fixed RNS afferents to the NTS and random synaptic weight values, the sympathetic efferents converged to the optimal values. When learning was completed, the output from the chemoreceptors became zero because the sympathetic efferents led to normal partial pressures of oxygen and carbon dioxide.</p> <p>Conclusions</p> <p>We introduce here a simple simulating computational theory to study, from a neurophysiologic point of view, the sympathetic development of cardiovascular regulation due to feedback signals sent off by cardiovascular receptors. The model simulates, too, how the NTS, as emergent property, acts as a comparator and how its rostral afferents behave as set point.</p

Neural set point for the control of arterial pressure: role of the nucleus tractus solitarius

<p>Abstract</p> <p>Background</p> <p>Physiological experiments have shown that the mean arterial blood pressure (MAP) can not be regulated after chemo and cardiopulmonary receptor denervation. Neuro-physiological information suggests that the nucleus tractus solitarius (NTS) is the only structure that receives information from its rostral neural nuclei and from the cardiovascular receptors and projects to nuclei that regulate the circulatory variables.</p> <p>Methods</p> <p>From a control theory perspective, to answer if the cardiovascular regulation has a set point, we should find out whether in the cardiovascular control there is something equivalent to a comparator evaluating the error signal (between the rostral projections to the NTS and the feedback inputs). The NTS would function as a comparator if: a) its lesion suppresses cardiovascular regulation; b) the negative feedback loop still responds normally to perturbations (such as mechanical or electrical) after cutting the rostral afferent fibers to the NTS; c) perturbation of rostral neural structures (RNS) to the NTS modifies the set point without changing the dynamics of the elicited response; and d) cardiovascular responses to perturbations on neural structures within the negative feedback loop compensate for much faster than perturbations on the NTS rostral structures.</p> <p>Results</p> <p>From the control theory framework, experimental evidence found currently in the literature plus experimental results from our group was put together showing that the above-mentioned conditions (to show that the NTS functions as a comparator) are satisfied.</p> <p>Conclusions</p> <p>Physiological experiments suggest that long-term blood pressure is regulated by the nervous system. The NTS functions as a comparator (evaluating the error signal) between its RNS and the cardiovascular receptor afferents and projects to nuclei that regulate the circulatory variables. The mean arterial pressure (MAP) is regulated by the feedback of chemo and cardiopulmonary receptors and the baroreflex would stabilize the short term pressure value to the prevailing carotid MAP. The discharge rates of rostral neural projections to the NTS would function as the set point of the closed and open loops of cardiovascular control. No doubt, then, the RNS play a functional role not only under steady-state conditions, but also in different behaviors and pathologies.</p

High mutual cooperation rates in rats learning reciprocal altruism: The role of payoff matrix

Cooperation is one of the most studied paradigms for the understanding of social interactions. Reciprocal altruism -a special type of cooperation that is taught by means of the iterated prisoner dilemma game (iPD)- has been shown to emerge in different species with different success rates. When playing iPD against a reciprocal opponent, the larger theoretical long-term reward is delivered when both players cooperate mutually. In this work, we trained rats in iPD against an opponent playing a Tit for Tat strategy, using a payoff matrix with positive and negative reinforcements, that is food and timeout respectively. We showed for the first time, that experimental rats were able to learn reciprocal altruism with a high average cooperation rate, where the most probable state was mutual cooperation (85%). Although when subjects defected, the most probable behavior was to go back to mutual cooperation. When we modified the matrix by increasing temptation rewards (T) or by increasing cooperation rewards (R), the cooperation rate decreased. In conclusion, we observe that an iPD matrix with large positive reward improves less cooperation than one with small rewards, shown that satisfying the relationship among iPD reinforcement was not enough to achieve high mutual cooperation behavior. Therefore, using positive and negative reinforcements and an appropriate contrast between rewards, rats have cognitive capacity to learn reciprocal altruism. This finding allows to infer that the learning of reciprocal altruism has early appeared in evolution.Fil: Delmas, Guillermo Ezequiel. Universidad de Buenos Aires. Facultad de Ingeniería; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Lew, Sergio Eduardo. Universidad de Buenos Aires. Facultad de Ingeniería; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Zanutto, Bonifacio Silvano. Universidad de Buenos Aires. Facultad de Ingeniería; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentin