A Computational Theory for the Learning of Equivalence Relations

Equivalence relations (ERs) are logical entities that emerge concurrently with the development of language capabilities. In this work we propose a computational model that learns to build ERs by learning simple conditional rules. The model includes visual areas, dopaminergic, and noradrenergic structures as well as prefrontal and motor areas, each of them modeled as a group of continuous valued units that simulate clusters of real neurons. In the model, lateral interaction between neurons of visual structures and top-down modulation of prefrontal/premotor structures over the activity of neurons in visual structures are necessary conditions for learning the paradigm. In terms of the number of neurons and their interaction, we show that a minimal structural complexity is required for learning ERs among conditioned stimuli. Paradoxically, the emergence of the ER drives a reduction in the number of neurons needed to maintain those previously specific stimulus–response learned rules, allowing an efficient use of neuronal resources

Cooperation in the iterated prisoner's dilemma is learned by operant conditioning mechanisms

The prisoner's dilemma (PD) is the leading metaphor for the evolution of cooperative behavior in populations of selfish agents. Although cooperation in the iterated prisoner's dilemma (IPD) has been studied for over twenty years, most of this research has been focused on strategies that involve nonlearned behavior. Another approach is to suppose that players' selection of the preferred reply might he enforced in the same way as feeding animals track the best way to feed in changing nonstationary environments. Learning mechanisms such as operant conditioning enable animals to acquire relevant characteristics of their environment in order to get reinforcements and to avoid punishments. In this study, the role of operant conditioning in the learning of cooperation was evaluated in the PD. We found that operant mechanisms allow the learning of IPD play against other strategies. When random moves are allowed in the game, the operant learning model showed low sensitivity. On the basis of this evidence, it is suggested that operant learning might be involved in reciprocal altruism.Fil: Gutnisky, D. A.. Universidad de Buenos Aires. Facultad de Ingenieria. Instituto de Ingeniería Biomédica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingenieria. Instituto de Ingeniería Biomédica; Argentin

Learning obstacle avoidance with an operant behavioral model

Artificial intelligence researchers have been attracted by the idea of having robots learn how to accomplish a task, rather than being told explicitly. Reinforcement learning has been proposed as an appealing framework to be used in controlling mobile agents. Robot learning research, as well as research in biological systems, face many similar problems in order to display high flexibility in performing a variety of tasks. In this work, the controlling of a vehicle in an avoidance task by a previously developed operant learning model (a form of animal learning) is studied. An environment in which a mobile robot with proximity sensors has to minimize the punishment for colliding against obstacles is simulated. The results were compared with the Q-Learning algorithm, and the proposed model had better performance. In this way a new artificial intelligence agent inspired by neurobiology, psychology, and ethology research is proposed.Fil: Gutnisky, D. A.. Universidad de Buenos Aires. Facultad de Ingeniería.Instituto de Ingeniería Biomédica; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingeniería.Instituto de Ingeniería Biomédica; Argentin

Probing the structure–function relationship with neural networks constructed by solving a system of linear equations

Neural network models are an invaluable tool to understand brain function since they allow us to connect the cellular and circuit levels with behaviour. Neural networks usually comprise a huge number of parameters, which must be chosen carefully such that networks reproduce anatomical, behavioural, and neurophysiological data. These parameters are usually fitted with off-the-shelf optimization algorithms that iteratively change network parameters and simulate the network to evaluate its performance and improve fitting. Here we propose to invert the fitting process by proceeding from the network dynamics towards network parameters. Firing state transitions are chosen according to the transition graph associated with the solution of a task. Then, a system of linear equations is constructed from the network firing states and membrane potentials, in a way that guarantees the consistency of the system. This allows us to uncouple the dynamical features of the model, like its neurons firing rate and correlation, from the structural features, and the task-solving algorithm implemented by the network. We employed our method to probe the structure–function relationship in a sequence memory task. The networks obtained showed connectivity and firing statistics that recapitulated experimental observations. We argue that the proposed method is a complementary and needed alternative to the way neural networks are constructed to model brain function.Fil: Mininni, Camilo Juan. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingeniería. Instituto de Ingeniería Biomédica.; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingeniería. Instituto de Ingeniería Biomédica.; Argentin

Bang-Bang Control of Feeding: Role of Hypothalamic and Satiety Signals

Rats, people, and many other omnivores eat in meals rather than continuously. We show by experimental test that eating in meals is regulated by a simple bang-bang control system, an idea foreshadowed by Le Magnen and many others, shown by us to account for a wide range of behavioral data, but never explicitly tested or tied to neurophysiological facts. The hypothesis is simply that the tendency to eat rises with time at a rate determined by satiety signals. When these signals fall below a set point, eating begins, in on–off fashion. The delayed sequelae of eating increment the satiety signals, which eventually turn eating off. Thus, under free conditions, the organism eats in bouts separated by noneating activities. We report an experiment with rats to test novel predictions about meal patterns that are not explained by existing homeostatic approaches. Access to food was systematically but unpredictably interrupted just as the animal tried to start a new meal. A simple bang-bang model fits the resulting meal-pattern data well, and its elements can be identified with neurophysiological processes. Hypothalamic inputs can provide the set point for longer-term regulation carried out by a comparator in the hindbrain. Delayed gustatory and gastrointestinal aftereffects of eating act via the nucleus of the solitary tract and other hindbrain regions as neural feedback governing short-term regulation. In this way, the model forges real links between a functioning feedback mechanism, neuro–hormonal data, and both short-term (meals) and long-term (eating-rate regulation) behavioral data

Blood pressure long term regulation: A neural network model of the set point development

<p>Abstract</p> <p>Background</p> <p>The notion of the nucleus tractus solitarius (NTS) as a comparator evaluating the error signal between its rostral neural structures (RNS) and the cardiovascular receptor afferents into it has been recently presented. From this perspective, stress can cause hypertension via set point changes, so offering an answer to an old question. Even though the local blood flow to tissues is influenced by circulating vasoactive hormones and also by local factors, there is yet significant sympathetic control. It is well established that the state of maturation of sympathetic innervation of blood vessels at birth varies across animal species and it takes place mostly during the postnatal period. During ontogeny, chemoreceptors are functional; they discharge when the partial pressures of oxygen and carbon dioxide in the arterial blood are not normal.</p> <p>Methods</p> <p>The model is a simple biological plausible adaptative neural network to simulate the development of the sympathetic nervous control. It is hypothesized that during ontogeny, from the RNS afferents to the NTS, the optimal level of each sympathetic efferent discharge is learned through the chemoreceptors' feedback. Its mean discharge leads to normal oxygen and carbon dioxide levels in each tissue. Thus, the sympathetic efferent discharge sets at the optimal level if, despite maximal drift, the local blood flow is compensated for by autoregulation. Such optimal level produces minimum chemoreceptor output, which must be maintained by the nervous system. Since blood flow is controlled by arterial blood pressure, the long-term mean level is stabilized to regulate oxygen and carbon dioxide levels. After development, the cardiopulmonary reflexes play an important role in controlling efferent sympathetic nerve activity to the kidneys and modulating sodium and water excretion.</p> <p>Results</p> <p>Starting from fixed RNS afferents to the NTS and random synaptic weight values, the sympathetic efferents converged to the optimal values. When learning was completed, the output from the chemoreceptors became zero because the sympathetic efferents led to normal partial pressures of oxygen and carbon dioxide.</p> <p>Conclusions</p> <p>We introduce here a simple simulating computational theory to study, from a neurophysiologic point of view, the sympathetic development of cardiovascular regulation due to feedback signals sent off by cardiovascular receptors. The model simulates, too, how the NTS, as emergent property, acts as a comparator and how its rostral afferents behave as set point.</p

Neural set point for the control of arterial pressure: role of the nucleus tractus solitarius

<p>Abstract</p> <p>Background</p> <p>Physiological experiments have shown that the mean arterial blood pressure (MAP) can not be regulated after chemo and cardiopulmonary receptor denervation. Neuro-physiological information suggests that the nucleus tractus solitarius (NTS) is the only structure that receives information from its rostral neural nuclei and from the cardiovascular receptors and projects to nuclei that regulate the circulatory variables.</p> <p>Methods</p> <p>From a control theory perspective, to answer if the cardiovascular regulation has a set point, we should find out whether in the cardiovascular control there is something equivalent to a comparator evaluating the error signal (between the rostral projections to the NTS and the feedback inputs). The NTS would function as a comparator if: a) its lesion suppresses cardiovascular regulation; b) the negative feedback loop still responds normally to perturbations (such as mechanical or electrical) after cutting the rostral afferent fibers to the NTS; c) perturbation of rostral neural structures (RNS) to the NTS modifies the set point without changing the dynamics of the elicited response; and d) cardiovascular responses to perturbations on neural structures within the negative feedback loop compensate for much faster than perturbations on the NTS rostral structures.</p> <p>Results</p> <p>From the control theory framework, experimental evidence found currently in the literature plus experimental results from our group was put together showing that the above-mentioned conditions (to show that the NTS functions as a comparator) are satisfied.</p> <p>Conclusions</p> <p>Physiological experiments suggest that long-term blood pressure is regulated by the nervous system. The NTS functions as a comparator (evaluating the error signal) between its RNS and the cardiovascular receptor afferents and projects to nuclei that regulate the circulatory variables. The mean arterial pressure (MAP) is regulated by the feedback of chemo and cardiopulmonary receptors and the baroreflex would stabilize the short term pressure value to the prevailing carotid MAP. The discharge rates of rostral neural projections to the NTS would function as the set point of the closed and open loops of cardiovascular control. No doubt, then, the RNS play a functional role not only under steady-state conditions, but also in different behaviors and pathologies.</p

High mutual cooperation rates in rats learning reciprocal altruism: The role of payoff matrix

Cooperation is one of the most studied paradigms for the understanding of social interactions. Reciprocal altruism -a special type of cooperation that is taught by means of the iterated prisoner dilemma game (iPD)- has been shown to emerge in different species with different success rates. When playing iPD against a reciprocal opponent, the larger theoretical long-term reward is delivered when both players cooperate mutually. In this work, we trained rats in iPD against an opponent playing a Tit for Tat strategy, using a payoff matrix with positive and negative reinforcements, that is food and timeout respectively. We showed for the first time, that experimental rats were able to learn reciprocal altruism with a high average cooperation rate, where the most probable state was mutual cooperation (85%). Although when subjects defected, the most probable behavior was to go back to mutual cooperation. When we modified the matrix by increasing temptation rewards (T) or by increasing cooperation rewards (R), the cooperation rate decreased. In conclusion, we observe that an iPD matrix with large positive reward improves less cooperation than one with small rewards, shown that satisfying the relationship among iPD reinforcement was not enough to achieve high mutual cooperation behavior. Therefore, using positive and negative reinforcements and an appropriate contrast between rewards, rats have cognitive capacity to learn reciprocal altruism. This finding allows to infer that the learning of reciprocal altruism has early appeared in evolution.Fil: Delmas, Guillermo Ezequiel. Universidad de Buenos Aires. Facultad de Ingeniería; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Lew, Sergio Eduardo. Universidad de Buenos Aires. Facultad de Ingeniería; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Zanutto, Bonifacio Silvano. Universidad de Buenos Aires. Facultad de Ingeniería; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentin

Transfer of artificial syntax functions among equivalence - related stimuli: An event - related brain potentials study

La formación de clases de equivalencia entre estímulos ha sido propuesta en el campo del Análisis Experimental del Comportamiento como un prerrequisito conductual para el lenguaje. Adicionalmente, existe evidencia de que la transferencia de función entre estímulos equivalentes podría explicar la adquisición de estructuras sintácticas simples. No obstante, la simplicidad de las funciones sintácticas estudiadas no capturó la complejidad combinatoria de la gramática natural. Si la transferencia de funciones en clases de equivalencia es un modelo posible del desarrollo de estructuras gramaticales, debería verificarse en contextos más válidos para el estudio del lenguaje. Los objetivos del presente trabajo fueron: 1) analizar la transferencia de funciones sintácticas en clases de equivalencia en un contexto válido para el estudio de la adquisición de reglas gramaticales, utilizando para ello el paradigma de gramáticas artificiales 2) Analizar los potenciales cerebrales relacionados con el procesamiento de esta transferencia de función, en secuencias gramaticales y no gramaticales. Se encontró evidencia comportamental de transferencia de función en un subgrupo de los sujetos experimentales. El potencial P600, típicamente asociado al costo de integración sintáctica en contextos linguísticos, fue observado en estos sujetos ante: violaciones gramaticales con estímulos originales de la gramática artificial y secuencias con estímulos relacionados por equivalencia (gramaticales y no gramaticales). Se interpretó que el procesamiento de las secuencias artificiales implicó mecanismos neurobiológicos similares a los asociados a la sintaxis del lenguaje, y que el patrón de actividad P600 observado puede ser explicado por el aumento del costo de integración de los estímulos al contexto previo.Stimulus equivalence class formation has been proposed as a behavioral prerequisite for language within the field of experimental analysis of behavior. Additionally, there is evidence that transfer of function among equivalent stimuli may explain acquisition of simple syntactic structures. However these experiments analyzed sequence functions that did not capture the complexity and versatility of natural grammar. If transfer of function between stimuli that belong to the same equivalence classes is indeed a useful model for the development of grammatical structures, then we should be able to verify it in a more valid context for the study of language. Artificial grammar learning tasks have been applied to the study of several aspects of language acquisition, from word segmentation to phrase structure and syntax rules. Furthermore, it has been shown that patterns of brain activity during processing of artificial grammars resemble those observed in language syntax processing. In particular, structural violations of language sentences and artificial grammar sequences both activate Broca's area. Therefore, artificial grammars provide a valid paradigm to study the learning of syntactic functions. The main objectives of the current work were: (1) to analyze transfer of function within equivalence classes in a valid context for the study of syntax acquisition, applying the artificial grammar paradigm and (2) to analyze brain potentials related to the transfer of function in grammatical and ungrammatical sequences. Fifteen subjects were trained to form two three-stimulus equivalence classes and then performed an artificial grammar learning task. One stimulus from each equivalence class was included as an item in the artificial grammar categories. During a test stage, subjects were asked to classify new artificial grammar sequences as grammatical or ungrammatical, while their EEG activity was registered. Half of these new sequences were built using the original training items and the other half contained equivalence-related stimulus. Subjects were assigned to two groups according to their performance in this test stage. Those participants whose percentage of correct responses was above 50 % were considered to pass, while those below were assigned to the fail group. We found behavioral evidence of transfer of function in the pass subgroup. These participants were able to correctly discriminate grammatical from un grammatical sequences that were built using original or equivalence-related stimulus. Event-Related potential Analysis of the EEG signal indicated a posteriorly distributed positivity with a topography and time-course similar to the P600 potential. Within linguistic contexts, P600 is interpreted as the neural correlate of prediction and integration costs during syntax processing. It has been proposed that sentence comprehension depends on predictive mechanisms that combine lexical, semantic and syntactic information from linguistic input to anticipate future words. Processing of incoming stimuli is facilitated by pre- activation, allowing rapid integration to previous context. However, when the input does not match predictions, this integration becomes slower and more difficult, requiring additional neural resources. The P600 has been considered and index of increased integration costs, generated by unfulfilled predictions of word category and morphology based on previous context. In the present experiment, the P600 was observed after: grammar violations with the original artificial grammar lexicon and artificial sequences containing equivalence-related stimulus (both grammatical andungrammatical). Results showed that artificial grammar processing involved neurobiological mechanisms that are similar to those associated in natural grammar processing. We interpreted the observed P600 pattern in terms of an increased stimulus integration cost, both in the case of grammatical and ungrammatical equivalence-related stimulus. Even though we consider that transference of function and equivalence class formations are by themselves insufficient to explain the complexity of natural grammar, we propose that this processes might be relevant to its acquisition and evolution, constituting a behavioral prerequisite for language development.Fil: Tabullo, Angel Javier. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Mendoza. Instituto de Ciencias Humanas, Sociales y Ambientales; ArgentinaFil: Yorio, Alberto. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires; ArgentinaFil: Zanutto, Silvano. Universidad de Buenos Aires. Facultad de Psicología; ArgentinaFil: Wainselboim, Alejandro Javier. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Mendoza. Instituto de Ciencias Humanas, Sociales y Ambientales; Argentin