On The Joint Modeling of The Behavior of Social Insects and Their Interaction With Environment by Taking Into Account Physical Phenomena Like Anisotropic Diffusion

International audienceThis work takes place in the framework of GEODIFF project (funded by CNRS) and deals with the general issue of the social behavior modeling of pest insects with a particular focus on Bark Beetles. Bark Beetles are responsible for pine trees devastation in North America since 2005. In order to stem the problem and to apply an adapted strategy, one should be able to predict the evolution of the population of Bark Beetles. More precisely, a model taking into account a given population of insects (a colony) interacting with its environment, the forest ecosystem, would be very helpful. In a previous work, we aimed to model diffusive phenomenons across the environment using a simple reactive Multi-agent System. Bark beetle use pheromones as a support for recruitment of other bark beetles in the neighborhood in order to achieve a mass attack over a tree. They are first attracted by the ethanol or other phytopheromones emitted by a sick, stressed or dead tree and reinforce the presence of other individuals amongst the targeted tree. Both ethanol and semiochemicals are transported through the forest thanks to the wind, thermic effects and this advection phenomenon is modulated by the topology of the environment, tree and other obstacles distribution. In other words, the environment is involved in the process of a bark beetle attack. The first modeling we used to tackle our objective was not spatially explicit as long as free space propagation only was taken into account (isotropic phenomenon) with no constraint imposed by the environment such as wind. This article is intended to take into account such physical phenomenons and push the modeling one step further by providing predictions driven by measures provided by a Geographical Information System

On The Joint Modeling of The Behavior of Social Insects and Their Interaction With Environment by Taking Into Account Physical Phenomena Like Anisotropic Diffusion

This work takes place in the framework of GEODIFF project (funded by CNRS) and deals with the general issue of the social behavior modeling of pest insects with a particular focus on Bark Beetles. Bark Beetles are responsible for pine trees devastation in North America since 2005. In order to stem the problem and to apply an adapted strategy, one should be able to predict the evolution of the population of Bark Beetles. More precisely, a model taking into account a given population of insects (a colony) interacting with its environment, the forest ecosystem, would be very helpful. In a previous work, we aimed to model diffusive phenomenons across the environment using a simple reactive Multi-agent System. Bark beetle use pheromones as a support for recruitment of other bark beetles in the neighborhood in order to achieve a mass attack over a tree. They are first attracted by the ethanol or other phytopheromones emitted by a sick, stressed or dead tree and reinforce the presence of other individuals amongst the targeted tree. Both ethanol and semiochemicals are transported through the forest thanks to the wind, thermic effects and this advection phenomenon is modulated by the topology of the environment, tree and other obstacles distribution. In other words, the environment is involved in the process of a bark beetle attack. The first modeling we used to tackle our objective was not spatially explicit as long as free space propagation only was taken into account (isotropic phenomenon) with no constraint imposed by the environment such as wind. This article is intended to take into account such physical phenomenons and push the modeling one step further by providing predictions driven by measures provided by a Geographical Information System

Un modèle de rejeu de séquences spatiales dans un réseau Hippocampe-Cortex préfrontal utilisant le reservoir computing

As rats learn to search for multiple sources of food or water in a complex environment, processes of spatial sequence learning and recall in the hippocampus (HC) and prefrontal cortex (PFC) are taking place. Recent studies (De Jong et al. 2011; Carr, Jadhav, and Frank 2011) show that spatial navigation in the rat hippocampus involves the replay of place-cell firing during awake and sleep states generating small contiguous subsequences of spatially related place-cell activations that we will call "snippets". These "snippets" occur primarily during sharp-wave-ripple (SPWR) events. Much attention has been paid to replay during sleep in the context of long-term memory consolidation. Here we focus on the role of replay during the awake state, as the animal is learning across multiple trials.We hypothesize that these "snippets" can be used by the PFC to achieve multi-goal spatial sequence learning.We propose to develop an integrated model of HC and PFC that is able to form place-cell activation sequences based on snippet replay. The proposed collaborative research will extend existing spatial cognition model for simpler goal-oriented tasks (Barrera and Weitzenfeld 2008; Barrera et al. 2015) with a new replay-driven model for memory formation in the hippocampus and spatial sequence learning and recall in PFC.In contrast to existing work on sequence learning that relies heavily on sophisticated learning algorithms and synaptic modification rules, we propose to use an alternative computational framework known as reservoir computing (Dominey 1995) in which large pools of prewired neural elements process information dynamically through reverberations. This reservoir computational model will consolidate snippets into larger place-cell activation sequences that may be later recalled by subsets of the original sequences.The proposed work is expected to generate a new understanding of the role of replay in memory acquisition in complex tasks such as sequence learning. That operational understanding will be leveraged and tested on a an embodied-cognitive real-time framework of a robot, related to the animat paradigm (Wilson 1991) [etc...]Alors que le rat apprend à chercher de multiples sources de nourriture ou d'eau, des processus d'apprentissage de séquences spatiales et de rejeu ont lieu dans l'hippocampe et le cortex préfrontal.Des études récentes (De Jong et al. 2011; Carr, Jadhav, and Frank 2011) mettent en évidence que la navigation spatiale dans l'hippocampe de rat implique le rejeu de l'activation de cellules de lieu durant les étant de sommeil et d'éveil en générant des petites sous séquences contigues d'activation de cellules de lieu cohérentes entre elles. Ces fragments sont observés en particulier lors d'évènements sharp wave ripple (SPWR).Les phénomènes de rejeu lors du sommeil dans le contexte de la consolidation de la mémoire à long terme ont beaucoup attiré l'attention. Ici nous nous focalisons sur le rôle du rejeu pendant l'état d'éveil.Nous formulons l'hypothèse que ces fragments peuvent être utilisés par le cortex préfrontal pour réaliser une tâche d'apprentissage spatial comprenant plusieurs buts.Nous proposons de développer un modèle intégré d'hippocampe et de cortex préfrontal capable de générer des séquences d'activation de cellules de lieu.Le travail collaboratif proposé prolonge les travaux existants sur un modèle de cognition spatiale pour des tâches orientés but plus simples (Barrera and Weitzenfeld 2008; Barrera et al. 2015) avec un nouveau modèle basé sur le rejeu pour la formation de mémoire dans l'hippocampe et l'apprentissage et génération de séquences spatiales par le cortex préfrontal.En contraste avec les travaux existants d'apprentissage de séquence qui repose sur des règles d'apprentissage sophistiquées, nous proposons d'utiliser un paradigme calculatoire appelé calcul par réservoir (Dominey 1995) dans lequel des groupes importants de neurones artificiels dont la connectivité est fixe traitent dynamiquement l'information au travers de réverbérations. Ce modèle calculatoire par réservoir consolide les fragments de séquence d'activations de cellule de lieu en une plus grande séquence qui pourra être rappelée elle-même par des fragments de séquence.Le travail proposé est supposé contribuer à une nouvelle compréhension du rôle du phénomène de rejeu dans l'acquisition de la mémoire dans une tâche complexe liée à l'apprentissage de séquence.Cette compréhension opérationnelle sera mise à profit et testée dans l'architecture cognitive incarnée d'un robot mobile selon l'approche animat (Wilson 1991) [etc...

panda-gym: Open-source goal-conditioned environments for robotic learning

This paper presents panda-gym, a set of Reinforcement Learning (RL) environments for the Franka Emika Panda robot integrated with OpenAI Gym. Five tasks are included: reach, push, slide, pick & place and stack. They all follow a Multi-Goal RL framework, allowing to use goal-oriented RL algorithms. To foster open-research, we chose to use the open-source physics engine PyBullet. The implementation chosen for this package allows to define very easily new tasks or new robots. This paper also presents a baseline of results obtained with state-of-the-art model-free off-policy algorithms. panda-gym is open-source and freely available at https://github.com/qgallouedec/panda-gym.Comment: NeurIPS 2021 Workshop on Robot Learning: Self-Supervised and Lifelong Learnin

Real-time sensory–motor integration of hippocampal place cell replay and prefrontal sequence learning in simulated and physical rat robots for novel path optimization

International audienceAn open problem in the cognitive dimensions of navigation concerns how previous exploratory experience is reorganized in order to allow the creation of novel efficient navigation trajectories. This behavior is revealed in the "traveling salesrat problem" (TSP) when rats discover the shortest path linking baited food wells after a few exploratory traversals. We have recently published a model of navigation sequence learning, where sharp wave ripple replay of hippocampal place cells transmit "snippets" of the recent trajectories that the animal has explored to the prefrontal cortex (PFC) (Cazin et al. in PLoS Comput Biol 15:e1006624, 2019). PFC is modeled as a recurrent reservoir network that is able to assemble these snippets into the efficient sequence (trajectory of spatial locations coded by place cell activation). The model of hippocampal replay generates a distribution of snippets as a function of their proximity to a reward, thus implementing a form of spatial credit assignment that solves the TSP task. The integrative PFC reservoir reconstructs the efficient TSP sequence based on exposure to this distribution of snippets that favors paths that are most proximal to rewards. While this demonstrates the theoretical feasibility of the PFC-HIPP interaction, the integration of such a dynamic system into a real-time sensory-motor system remains a challenge. In the current research, we test the hypothesis that the PFC reservoir model can operate in a real-time sensory-motor loop. Thus, the main goal of the paper is to validate the model in simulated and real robot scenarios. Place cell activation encoding the current position of the simulated and physical rat robot feeds the PFC reservoir which generates the successor place cell activation that represents the next step in the reproduced sequence in the readout. This is input to the robot, which advances to the coded location and then generates de novo the current place cell activation. This allows demonstration of the crucial role of embodiment. If the spatial code readout from PFC is played back directly into PFC, error can accumulate, and the system can diverge from desired trajectories. This required a spatial filter to decode the PFC code to a location and then recode a new place cell code for that location. In the robot, the place cell vector output of PFC is used to physically displace the robot and then generate a new place cell coded input to the PFC, replacing part of the software recoding procedure that was required otherwise. We demonstrate how this integrated sensory-motor system can learn simple navigation sequences and then, importantly, how it can synthesize novel efficient sequences based on prior experience, as previously demonstrated (Cazin et al. 2019). This contributes to the understanding of hippocampal replay in novel navigation sequence formation and the important role of embodiment.Prefrontal cortex; Hippocampus; Navigation; Replay; Sharp wave ripple; Reservoir computing; Traveling sales person; Reinforcement learning ;Spatial Cognition; Memory Consolidation; Natural-Language; Reverse Replay; Grid Cells; Model; Reactivation; Cortex; State; Representatio

Favoriser l’équité dans les classes caractérisées par une forte diversité linguistique : recours aux activités plurilingues dans une perspective coopérative

Cette recherche-action a été développée en collaboration avec une enseignante d’une classe de 3e primaire (7-8 ans) afin de soutenir la participation de tous les élèves dans une classe multiculturelle et plurilingue. La stratégie pédagogique implantée porte une attention particulière aux élèves peu choisis comme camarades de jeu ou partenaires de travail – autrement dit, qui ont un bas statut parmi leurs pairs. L’objectif du dispositif est de valoriser la contribution de ces élèves en s’appuyant sur leurs compétences linguistiques en langue d’origine dans des activités coopératives. L’étude mesure l’évolution de leur statut parmi les pairs (le nombre de fois où ils sont choisis pour jouer ou travailler) et évalue dans quelle mesure ce statut prédit les interactions dans les petits groupes avant et après l’intervention. Avant l’intervention, ceux qui ont un statut plus élevé dominent les interactions, indiquant une participation inégale. Suite à l’intervention, les statuts et la participation s’améliorent pour tous, et particulièrement pour les élèves qui avaient un statut initial bas. Le statut ne prédit alors plus les interactions dans les groupes de travail. Ces résultats ouvrent des pistes pour favoriser l’équité dans les classes à forte diversité linguistique.This action research was developed in collaboration with a teacher from a grade 3 classroom (7-8 year-olds) to support the participation of all students in a multicultural and multilingual classroom. The pedagogical strategy focuses on students who are not often chosen as playmates or work partners - those who have low status among their peers. The aim of the strategy is to valorize their contributions through cooperative activities that encourage them to use their maternal language skills. The study measures the development of their status among their peers (the number of times they are chosen to play or work) and assesses how well this status predicts interactions in small groups before and after the intervention. Prior to the intervention, those with higher status dominated interactions, indicating unequal participation. Following the intervention, status and participation improved for everyone, especially for students with low initial status. Status therefore no longer predicted interactions in the workgroups. These results open up avenues for promoting equity in classrooms with high linguistic diversity.Esta investigación-acción se desarrolló con la colaboración de una maestra de 3º de primaria (7-8 años) con la finalidad de fomentar la participación de todos los alumnos de una clase multicultural y plurilingüe. La estrategia pedagógica implantada puso una atención particular a los alumnos que raramente eran escogidos como camaradas de juego o coparticipes de trabajo –es decir que tenían bajo estatus entre sus pares. El objetivo del dispositivo es valorizar la contribución de esos alumnos apoyándose en sus competencias lingüísticas en lengua materna en las actividades cooperativas. El estudio midió la evolución de su estatus entre sus pares (el número de veces que fueron escogidos para jugar o trabajar) y evaluó en qué medida dicho estatus predice de las interacciones en los pequeños grupos antes y después de la intervención. Antes de la intervención, quienes tenían un estatus alto dominaban las interacciones, indicando una participación desigual. Después de la intervención, todos mejoraron sus estatus y su participación, particularmente los alumnos que inicialmente gozaban de un bajo estatus. Así pues, el estatus ya no predice las interacciones en los grupos de trabajo. Estos resultados abren vías para fomentar la igualdad en las clases con fuerte diversidad lingüística