Modeling pavlovian conditioning with multiple neuronal populations

International audienceArtificial Neural Networks are often used as black boxes to implement behavioral functions, developed by trials and errors, fed with sensory inputs and controlled by some criteria of performance. This is the case for pavlovian conditioning where important sensory information is non ambiguous and where the error of prediction is to be minimized. These past years, taking into account critical conditioning behaviors entailed complexifying the neuronal functioning and learning rules. This resulted in networks still simple at the architectural level but with a dynamics difficult to master. Instead, we propose a new neuronal model using uniform and classical neuronal dynamics, with a more complex architecture based on recent findings in neuroscience. Results reported in this paper confirm the good behavior of the model and justify the complex architecture by the greater robustness and flexibility of the model

A System-Level Model of Noradrenergic Function

International audienceNeuromodulation is an interesting way to display different modes of functioning in a complex network. The effect of Noradrenaline has often been related to the exploration/exploitation trade-off and implemented in models by modulation of the gain of activation function. In this paper, we show that this mechanism is not sufficient for system-level networks and propose another way to implement it, exploiting reported inhibition of a striatal region by Noradrenaline. We describe here the corresponding model and report its performances in a reversal task

Modeling Neuromodulation as a Framework to Integrate Uncertainty in General Cognitive Architectures

International audienceOne of the most critical properties of a versatile intelligent agent is its capacity to adapt autonomously to any change in the environment without overly complexifying its cognitive architecture. In this paper, we propose that understanding the role of neuromodulation in the brain is of central interest for this purpose. More precisely, we propose that an accurate estimation of the nature of uncertainty present in the environment is performed by specific brain regions and broadcast throughout the cerebral network by neuromodulators, resulting in appropriate changes in cerebral functioning and learning modes. Better understanding the principles of these mechanisms in the brain might tremendously inspire the field of Artificial General Intelligence. The original contribution of this paper is to relate the four major neuromodulators to four fundamental dimensions of uncertainty

Modeling the sensory roles of noradrenaline in action selection

International audienceNoradrenaline participates in the neuromodulation of brain activity to modify the trade-off between exploration and exploitation when sensory contingencies have changed. Accordingly, attentional models of noradrenaline acting on sensory representations have been proposed. In this paper , we explore another possible action of this neuromodulator in the decision making process and report simulation results that illustrate that its role is concerned with different aspects of sensory processing. This is made possible by the extension of a classical model of action selection, to render it able to detect and to adapt to sudden changes in sensory contingencies, which is a major characteristic of autonomous learning

Émergence de catégories par interaction entre systèmes d'apprentissage

National audienceLe modèle AGREL est un modèle connexionniste de catégorisation, reposant sur des observations comportementales et physiologiques qui indiquent que la création de catégories perceptives se fonde sur des phénomènes attentionnels et sur des critères d'erreur de prédiction de récompense. Ce modèle d'apprentissage par renforcement laisse ouverte la discussion sur l'origine du signal de récompense. Il impose également la supervision de la couche de sortie par la tâche de sélection de l'action. Le modèle que nous proposons étend ces travaux dans deux directions. D'une part, nous reposant sur des données biologiques, nous introduisons un réseau spécifique pour le calcul de l'erreur de prédiction, préfigurant l'action de l'amygdale. D'autre part, ce calcul dans un module séparé permet d'adapter le modèle AGREL spécifiquement pour la création des catégories nécessaires à la tâche, indépendemment de la sélection de l'action. Cette approche, dont la performance est évaluée par des tests classiques de discrimination, illustre la puissance de la structuration modulaire du cerveau, où la spécialisation de structures distinctes à des formes différentes d'apprentissage permet un plus grand pouvoir expressif et une grande flexibilité dans l'analyse et la représentation de l'information

Using the Amygdala in decision making

International audienceDecision making is often described as composed of multiple loops, mainly the limbic, associative, and motor loops, in the Prefrontal Cortex and Basal Ganglia. While the various nuclei of the Amygdala has been traditionaly considered for their role in fear prediction and respondent conditioning [9, 4, 7], structural similitudes have been reported between the central amygdala (CeA) and structures involved in decision making the nucleus accumbens and the pallidum [5]. Particularly, the lateral capsular, lateral and medial subdivisions of CeA possess similarities in structures and connectivity respectively with the shell, the core of the nucleus accumbens and the pallidum. This, along with a spatial continuity between CeA and the shell of the nucleus accumbens [5], leads to the hypothesis that respondant conditioning could be seen as a loop more primitive but similar to decision-making loops. Moreover, lesions of the amygdala, and especially of the basal nucleus of the amygdala, impair operant conditioning paradigms like devaluation or reversal [8], or decision making in gambling [1]. In a direct way, learning associations between CS (conditioned stimuli) and US (unconditioned stimuli, ie. reward or punishment) allows the amygdala to learn CS values [2], and to provide such values in to the OFC and ventral striatum for goal-directed behaviors [8]. In an indirect way, the amygdala projects to VTA-SNc for dopamine and to the basal forebrain for acetylcholine, thus providing indirect reinforcing signals to the decision making system. We present here a simple neuronal model of the amygdala and propose to compare it to the decision making loops. Our model is composed of five populations from three different amygdalar nuclei. Neurons in these populations are described using a classical mean-rate formalism and a sigmoid activation function. Learning is hebbian and uses a Rescorla-Wagner like prediction error. One specificity is that amygdalar activation also takes into account the effect of acetylcholine, which modulates the competition between different amygdalar populations for chosing a sensory-based rule or a contextual one [11, 10]. Acetylcholine concentration is computed from the recent prediction errors of our model, and as such reflects the known uncertainty in reward prediction [12]. This model successfully reproduces experimental data recorded in fear and extinction conditioning [7], along with the effect of pathways impairment as reported in [11, 10, 3]. This model is the first step in modeling operant conditioning and goal-directed behavior. Thus, ongoing work is to extend this model to operant conditioning, by including OFC, shell and ventral pallidum structures. Another work in progress proposes to use the uncertainty level computed by our amygdalar network to help a decision making system to choose between exploration and exploitation. If known uncertainty is low, it means the model is correctly predicting the rule, so it should favour exploitation. At the opposite, the higher the uncertainty in predicting US, the more should it explore, because its current strategy is not reliable. We propose to highlight here both the similarities in functioning and reciprocal influences between respondent conditioning, as performed by our amygdalar model, and decision making

A biologically inspired neuronal model of reward prediction error computation

International audienceThe neurocomputational model described here proposes that two dimensions involved in computation of reward prediction errors i.e magnitude and time could be computed separately and later combined unlike traditional reinforcement learning models. The model is built on biological evidences and is able to reproduce various aspects of classical conditioning, namely, the progressive cancellation of the predicted reward, the predictive firing from conditioned stimuli, and delineation of early rewards by showing firing for sooner early rewards and not for early rewards that occur with a longer latency in accordance with biological data

From biological to numerical experiments in systemic neuroscience: a simulation platform

International audienceStudying and modeling the brain as a whole is a real challenge. For such systemic models (in contrast to models of one brain area or aspect), there is a real need for new tools designed to perform complex numerical experiments, beyond usual tools distributed in the computer science and neuroscience communities. Here, we describe an effective solution, freely available on line and already in use, to validate such models of the brain functions. We explain why this is the best choice, as a complement to robotic setup, and what are the general requirements for such a benchmarking platform. In this experimental setup, the brainy-bot implementing the model to study is embedded in a simplified but realistic controlled environment. From visual, tactile and olfactory input, to body, arm and eye motor command, in addition to vital interoceptive cues, complex survival behaviors can be experimented. We also discuss here algorithmic high-level cognitive modules, making the job of building biologically plausible bots easier. The key point is to possibly alternate the use of symbolic representation and of complementary and usual neural coding. As a consequence, algorithmic principles have to be considered at higher abstract level, beyond a given data representation, which is an interesting challenge

Altimetry for the future: Building on 25 years of progress

In 2018 we celebrated 25 years of development of radar altimetry, and the progress achieved by this methodology in the fields of global and coastal oceanography, hydrology, geodesy and cryospheric sciences. Many symbolic major events have celebrated these developments, e.g., in Venice, Italy, the 15th (2006) and 20th (2012) years of progress and more recently, in 2018, in Ponta Delgada, Portugal, 25 Years of Progress in Radar Altimetry. On this latter occasion it was decided to collect contributions of scientists, engineers and managers involved in the worldwide altimetry community to depict the state of altimetry and propose recommendations for the altimetry of the future. This paper summarizes contributions and recommendations that were collected and provides guidance for future mission design, research activities, and sustainable operational radar altimetry data exploitation. Recommendations provided are fundamental for optimizing further scientific and operational advances of oceanographic observations by altimetry, including requirements for spatial and temporal resolution of altimetric measurements, their accuracy and continuity. There are also new challenges and new openings mentioned in the paper that are particularly crucial for observations at higher latitudes, for coastal oceanography, for cryospheric studies and for hydrology. The paper starts with a general introduction followed by a section on Earth System Science including Ocean Dynamics, Sea Level, the Coastal Ocean, Hydrology, the Cryosphere and Polar Oceans and the ‘‘Green” Ocean, extending the frontier from biogeochemistry to marine ecology. Applications are described in a subsequent section, which covers Operational Oceanography, Weather, Hurricane Wave and Wind Forecasting, Climate projection. Instruments’ development and satellite missions’ evolutions are described in a fourth section. A fifth section covers the key observations that altimeters provide and their potential complements, from other Earth observation measurements to in situ data. Section 6 identifies the data and methods and provides some accuracy and resolution requirements for the wet tropospheric correction, the orbit and other geodetic requirements, the Mean Sea Surface, Geoid and Mean Dynamic Topography, Calibration and Validation, data accuracy, data access and handling (including the DUACS system). Section 7 brings a transversal view on scales, integration, artificial intelligence, and capacity building (education and training). Section 8 reviews the programmatic issues followed by a conclusion

Altimetry for the future: building on 25 years of progress

In 2018 we celebrated 25 years of development of radar altimetry, and the progress achieved by this methodology in the fields of global and coastal oceanography, hydrology, geodesy and cryospheric sciences. Many symbolic major events have celebrated these developments, e.g., in Venice, Italy, the 15th (2006) and 20th (2012) years of progress and more recently, in 2018, in Ponta Delgada, Portugal, 25 Years of Progress in Radar Altimetry. On this latter occasion it was decided to collect contributions of scientists, engineers and managers involved in the worldwide altimetry community to depict the state of altimetry and propose recommendations for the altimetry of the future. This paper summarizes contributions and recommendations that were collected and provides guidance for future mission design, research activities, and sustainable operational radar altimetry data exploitation. Recommendations provided are fundamental for optimizing further scientific and operational advances of oceanographic observations by altimetry, including requirements for spatial and temporal resolution of altimetric measurements, their accuracy and continuity. There are also new challenges and new openings mentioned in the paper that are particularly crucial for observations at higher latitudes, for coastal oceanography, for cryospheric studies and for hydrology. The paper starts with a general introduction followed by a section on Earth System Science including Ocean Dynamics, Sea Level, the Coastal Ocean, Hydrology, the Cryosphere and Polar Oceans and the “Green” Ocean, extending the frontier from biogeochemistry to marine ecology. Applications are described in a subsequent section, which covers Operational Oceanography, Weather, Hurricane Wave and Wind Forecasting, Climate projection. Instruments’ development and satellite missions’ evolutions are described in a fourth section. A fifth section covers the key observations that altimeters provide and their potential complements, from other Earth observation measurements to in situ data. Section 6 identifies the data and methods and provides some accuracy and resolution requirements for the wet tropospheric correction, the orbit and other geodetic requirements, the Mean Sea Surface, Geoid and Mean Dynamic Topography, Calibration and Validation, data accuracy, data access and handling (including the DUACS system). Section 7 brings a transversal view on scales, integration, artificial intelligence, and capacity building (education and training). Section 8 reviews the programmatic issues followed by a conclusion