A Heteroassociative Learning Model Robust to Interference

Best Paper AwardInternational audienceNeuronal models of associative memories are recurrent networks able to learn quickly patterns as stable states of the network. Their main acknowledged weakness is related to catastrophic interference when too many or too close examples are stored. Based on biological data we have recently proposed a model resistant to some kinds of interferences related to heteroassociative learning. In this paper we report numerical experiments that highlight this robustness and demonstrate very good performances of memorization. We also discuss convergence of interests for such an adaptive mechanism for biological modeling and information processing in the domain of machine learning

Integration of exteroceptive and interoceptive information within the hippocampus: a computational study

International audienceCitation: Kassab R and Alexandre F (2015) Integration of exteroceptive and interoceptive information within the hippocampus: a computational study. Front. Syst. Neurosci. 9:87. Many episodic memory studies have critically implicated the hippocampus in the rapid binding of sensory information from the perception of the external environment, reported by exteroception. Other structures in the medial temporal lobe, especially the amygdala, have been more specifically linked with emotional dimension of episodic memories, reported by interoception. The hippocampal projection to the amygdala is proposed as a substrate important for the formation of extero-interoceptive associations, allowing adaptive behaviors based on past experiences. Recently growing evidence suggests that hippocampal activity observed in a wide range of behavioral tasks could reflect associations between exteroceptive patterns and their emotional valences. The hippocampal computational models, therefore, need to be updated to elaborate better interpretation of hippocampal-dependent behaviors. In earlier models, interoceptive features, if not neglected, are bound together with other exteroceptive features through autoassociative learning mechanisms. This way of binding integrates both kinds of features at the same level, which is not always suitable for example in the case of pattern completion. Based on the anatomical and functional heterogeneity along the septotemporal and transverse axes of the hippocampus, we suggest instead that distinct hippocampal subregions may be engaged in the representation of these different types of information, each stored apart in autoassociative memories but linked together in a heteroassociative way. The model is developed within the hard constraint of rapid, even single trial, learning of episodic memories. The performance of the model is assessed quantitatively and its resistance to interference is demonstrated through a series of numerical experiments. An experiment of reversal learning in patients with amnesic cognitive impairment is also reproduced

Young LMC clusters: the role of red supergiants and multiple stellar populations in their integrated light and CMDs

The optical integrated spectra of three LMC young stellar clusters (NGC 1984, NGC 1994 and NGC 2011) exhibit concave continua and prominent molecular bands which deviate significantly from the predictions of single stellar population (SSP) models. In order to understand the appearance of these spectra, we create a set of young stellar population (MILES) models, which we make available to the community. We use archival International Ultraviolet Explorer integrated UV spectra to independently constrain the cluster masses and extinction, and rule out strong stochastic effects in the optical spectra. In addition, we also analyze deep colour-magnitude diagrams of the clusters to provide independent age determinations based on isochrone fitting. We explore hypotheses including age-spreads in the clusters, a top-heavy initial mass function, different SSP models and the role of red supergiant stars (RSG). We find that the strong molecular features in the optical spectra can only be reproduced by modeling an increased fraction of about 20 per cent by luminosity of RSG above what is predicted by canonical stellar evolution models. Given the uncertainties in stellar evolution at Myr ages, we cannot presently rule-out the presence of Myr age-spreads in these clusters. Our work combines different wavelengths as well as different approaches (resolved data as well as integrated spectra for the same sample) in order to reveal the complete picture. We show that each approach provides important information but in combination can we better understand the cluster stellar populations.Comment: Accepted for publication in MNRA

Analysis of stationary and emerging properties in information flows changing over time

De nombreuses applications génèrent et reçoivent des données sous la forme de flux continu, illimité, et très rapide. Cela pose naturellement des problèmes de stockage, de traitement et d'analyse de données qui commencent juste à être abordés dans le domaine des flux de données. Il s'agit, d'une part, de pouvoir traiter de tels flux à la volée sans devoir mémoriser la totalité des données et, d'autre part, de pouvoir traiter de manière simultanée et concurrente l'analyse des régularités inhérentes au flux de données et celle des nouveautés, exceptions, ou changements survenant dans ce même flux au cours du temps. L'apport de ce travail de thèse réside principalement dans le développement d'un modèle d'apprentissage - nommé ILoNDF - fondé sur le principe de la détection de nouveauté. L'apprentissage de ce modèle est, contrairement à sa version de départ, guidé non seulement par la nouveauté qu'apporte une donnée d'entrée mais également par la donnée elle-même. De ce fait, le modèle ILoNDF peut acquérir constamment de nouvelles connaissances relatives aux fréquences d'occurrence des données et de leurs variables, ce qui le rend moins sensible au bruit. De plus, doté d'un fonctionnement en ligne sans répétition d'apprentissage, ce modèle répond aux exigences les plus fortes liées au traitement des flux de données. Dans un premier temps, notre travail se focalise sur l'étude du comportement du modèle ILoNDF dans le cadre général de la classification à partir d'une seule classe en partant de l'exploitation des données fortement multidimensionnelles et bruitées. Ce type d'étude nous a permis de mettre en évidence les capacités d'apprentissage pures du modèle ILoNDF vis-à-vis de l'ensemble des méthodes proposées jusqu'à présent. Dans un deuxième temps, nous nous intéressons plus particulièrement à l'adaptation fine du modèle au cadre précis du filtrage d'informations. Notre objectif est de mettre en place une stratégie de filtrage orientée-utilisateur plutôt qu'orientée-système, et ceci notamment en suivant deux types de directions. La première direction concerne la modélisation utilisateur à l'aide du modèle ILoNDF. Cette modélisation fournit une nouvelle manière de regarder le profil utilisateur en termes de critères de spécificité, d'exhaustivité et de contradiction. Ceci permet, entre autres, d'optimiser le seuil de filtrage en tenant compte de l'importance que pourrait donner l'utilisateur à la précision et au rappel. La seconde direction, complémentaire de la première, concerne le raffinement des fonctionnalités du modèle ILoNDF en le dotant d'une capacité à s'adapter à la dérive du besoin de l'utilisateur au cours du temps. Enfin, nous nous attachons à la généralisation de notre travail antérieur au cas où les données arrivant en flux peuvent être réparties en classes multiples.Many applications produce and receive continuous, unlimited, and high-speed data streams. This raises obvious problems of storage, treatment and analysis of data, which are only just beginning to be treated in the domain of data streams. On the one hand, it is a question of treating data streams on the fly without having to memorize all the data. On the other hand, it is also a question of analyzing, in a simultaneous and concurrent manner, the regularities inherent in the data stream as well as the novelties, exceptions, or changes occurring in this stream over time. The main contribution of this thesis concerns the development of a new machine learning approach - called ILoNDF - which is based on novelty detection principle. The learning of this model is, contrary to that of its former self, driven not only by the novelty part in the input data but also by the data itself. Thereby, ILoNDF can continuously extract new knowledge relating to the relative frequencies of the data and their variables. This makes it more robust against noise. Being operated in an on-line mode without repeated training, ILoNDF can further address the primary challenges for managing data streams. Firstly, we focus on the study of ILoNDF's behavior for one-class classification when dealing with high-dimensional noisy data. This study enabled us to highlight the pure learning capacities of ILoNDF with respect to the key classification methods suggested until now. Next, we are particularly involved in the adaptation of ILoNDF to the specific context of information filtering. Our goal is to set up user-oriented filtering strategies rather than system-oriented in following two types of directions. The first direction concerns user modeling relying on the model ILoNDF. This provides a new way of looking at user's need in terms of specificity, exhaustivity and contradictory profile-contributing criteria. These criteria go on to estimate the relative importance the user might attach to precision and recall. The filtering threshold can then be adjusted taking into account this knowledge about user's need. The second direction, complementary to the first one, concerns the refinement of ILoNDF's functionality in order to confer it the capacity of tracking drifting user's need over time. Finally, we consider the generalization of our previous work to the case where streaming data can be divided into multiple classes

Analyse des propriétés stationnaires et des propriétés émergentes dans les flux d'informations changeant au cours du temps

Many applications produce and receive continuous, unlimited, and high-speed data streams. This raises obvious problems of storage, treatment and analysis of data, which are only just beginning to be treated in the domain of data streams. On the one hand, it is a question of treating data streams on the fly without having to memorize all the data. On the other hand, it is also a question of analyzing, in a simultaneous and concurrent manner, the regularities inherent in the data stream as well as the novelties, exceptions, or changes occurring in this stream over time.The main contribution of this thesis concerns the development of a new machine learning approach - called ILoNDF - which is based on novelty detection principle. The learning of this model is, contrary to that of its former self, driven not only by the novelty part in the input data but also by the data itself. Thereby, ILoNDF can continuously extract new knowledge relating to the relative frequencies of the data and their variables. This makes it more robust against noise. Being operated in an on-line mode without repeated training, ILoNDF can further address the primary challenges for managing data streams. Firstly, we focus on the study of ILoNDF's behavior for one-class classification when dealing with high-dimensional noisy data. This study enabled us to highlight the pure learning capacities of ILoNDF with respect to the key classification methods suggested until now. Next, we are particularly involved in the adaptation of ILoNDF to the specific context of information filtering. Our goal is to set up user-oriented filtering strategies rather than system-oriented in following two types of directions. The first direction concerns user modeling relying on the model ILoNDF. This provides a new way of looking at user's need in terms of specificity, exhaustivity and contradictory profile-contributing criteria. These criteria go on to estimate the relative importance the user might attach to precision and recall. The filtering threshold can then be adjusted taking into account this knowledge about user's need. The second direction, complementary to the first one, concerns the refinement of ILoNDF's functionality in order to confer it the capacity of tracking drifting user's need over time. Finally, we consider the generalization of our previous work to the case where streaming data can be divided into multiple classes.De nombreuses applications génèrent et reçoivent des données sous la forme de flux continu, illimité, et très rapide. Cela pose naturellement des problèmes de stockage, de traitement et d'analyse de données qui commencent juste à être abordés dans le domaine des flux de données. Il s'agit, d'une part, de pouvoir traiter de tels flux à la volée sans devoir mémoriser la totalité des données et, d'autre part, de pouvoir traiter de manière simultanée et concurrente l'analyse des régularités inhérentes au flux de données et celle des nouveautés, exceptions, ou changements survenant dans ce même flux au cours du temps.L'apport de ce travail de thèse réside principalement dans le développement d'un modèle d'apprentissage - nommé ILoNDF - fondé sur le principe de la détection de nouveauté. L'apprentissage de ce modèle est, contrairement à sa version de départ, guidé non seulement par la nouveauté qu'apporte une donnée d'entrée mais également par la donnée elle-même. De ce fait, le modèle ILoNDF peut acquérir constamment de nouvelles connaissances relatives aux fréquences d'occurrence des données et de leurs variables, ce qui le rend moins sensible au bruit. De plus, doté d'un fonctionnement en ligne sans répétition d'apprentissage, ce modèle répond aux exigences les plus fortes liées au traitement des flux de données. Dans un premier temps, notre travail se focalise sur l'étude du comportement du modèle ILoNDF dans le cadre général de la classification à partir d'une seule classe en partant de l'exploitation des données fortement multidimensionnelles et bruitées. Ce type d'étude nous a permis de mettre en évidence les capacités d'apprentissage pures du modèle ILoNDF vis-à-vis de l'ensemble des méthodes proposées jusqu'à présent. Dans un deuxième temps, nous nous intéressons plus particulièrement à l'adaptation fine du modèle au cadre précis du filtrage d'informations. Notre objectif est de mettre en place une stratégie de filtrage orientée-utilisateur plutôt qu'orientée-système, et ceci notamment en suivant deux types de directions. La première direction concerne la modélisation utilisateur à l'aide du modèle ILoNDF. Cette modélisation fournit une nouvelle manière de regarder le profil utilisateur en termes de critères de spécificité, d'exhaustivité et de contradiction. Ceci permet, entre autres, d'optimiser le seuil de filtrage en tenant compte de l'importance que pourrait donner l'utilisateur à la précision et au rappel. La seconde direction, complémentaire de la première, concerne le raffinement des fonctionnalités du modèle ILoNDF en le dotant d'une capacité à s'adapter à la dérive du besoin de l'utilisateur au cours du temps. Enfin, nous nous attachons à la généralisation de notre travail antérieur au cas où les données arrivant en flux peuvent être réparties en classes multiples

Analyse des propriétés stationnaires et des propriétés émergentes dans les flux d'information changeant au cours du temps

De nombreuses applications génèrent et reçoivent des données sous la forme de flux continu, illimité, et très rapide. Cela pose naturellement des problèmes de stockage, de traitement et d'analyse de données qui commencent juste à être abordés dans le domaine des flux de données. Il s'agit, d'une part, de pouvoir traiter de tels flux à la volée sans devoir mémoriser la totalité des données et, d'autre part, de pouvoir traiter de manière simultanée et concurrente l'analyse des régularités inhérentes au flux de données et celle des nouveautés, exceptions, ou changements survenant dans ce même flux au cours du temps. L'apport de ce travail de thèse réside principalement dans le développement d'un modèle d'apprentissage - nommé ILoNDF - fondé sur le principe de la détection de nouveauté. L'apprentissage de ce modèle est, contrairement à sa version de départ, guidé non seulement par la nouveauté qu'apporte une donnée d'entrée mais également par la donnée elle-même. De ce fait, le modèle ILoNDF peut acquérir constamment de nouvelles connaissances relatives aux fréquences d'occurrence des données et de leurs variables, ce qui le rend moins sensible au bruit. De plus, doté d'un fonctionnement en ligne sans répétition d'apprentissage, ce modèle répond aux exigences les plus fortes liées au traitement des flux de données. Dans un premier temps, notre travail se focalise sur l'étude du comportement du modèle ILoNDF dans le cadre général de la classification à partir d'une seule classe en partant de l'exploitation des données fortement multidimensionnelles et bruitées. Ce type d'étude nous a permis de mettre en évidence les capacités d'apprentissage pures du modèle ILoNDF vis-à-vis de l'ensemble des méthodes proposées jusqu'à présent. Dans un deuxième temps, nous nous intéressons plus particulièrement à l'adaptation fine du modèle au cadre précis du filtrage d'informations. Notre objectif est de mettre en place une stratégie de filtrage orientée-utilisateur plutôt qu'orientée-système, et ceci notamment en suivant deux types de directions. La première direction concerne la modélisation utilisateur à l'aide du modèle ILoNDF. Cette modélisation fournit une nouvelle manière de regarder le profil utilisateur en termes de critères de spécificité, d'exhaustivité et de contradiction. Ceci permet, entre autres, d'optimiser le seuil de filtrage en tenant compte de l'importance que pourrait donner l'utilisateur à la précision et au rappel. La seconde direction, complémentaire de la première, concerne le raffinement des fonctionnalités du modèle ILoNDF en le dotant d'une capacité à s'adapter à la dérive du besoin de l'utilisateur au cours du temps. Enfin, nous nous attachons à la généralisation de notre travail antérieur au cas où les données arrivant en flux peuvent être réparties en classes multiples.Many applications produce and receive continuous, unlimited, and high-speed data streams. This raises obvious problems of storage, treatment and analysis of data, which are only just beginning to be treated in the domain of data streams. On the one hand, it is a question of treating data streams on the fly without having to memorize all the data. On the other hand, it is also a question of analyzing, in a simultaneous and concurrent manner, the regularities inherent in the data stream as well as the novelties, exceptions, or changes occurring in this stream over time. The main contribution of this thesis concerns the development of a new machine learning approach - called ILoNDF - which is based on novelty detection principle. The learning of this model is, contrary to that of its former self, driven not only by the novelty part in the input data but also by the data itself. Thereby, ILoNDF can continuously extract new knowledge relating to the relative frequencies of the data and their variables. This makes it more robust against noise. Being operated in an on-line mode without repeated training, ILoNDF can further address the primary challenges for managing data streams. Firstly, we focus on the study of ILoNDF's behavior for one-class classification when dealing with high-dimensional noisy data. This study enabled us to highlight the pure learning capacities of ILoNDF with respect to the key classification methods suggested until now. Next, we are particularly involved in the adaptation of ILoNDF to the specific context of information filtering. Our goal is to set up user-oriented filtering strategies rather than system-oriented in following two types of directions. The first direction concerns user modeling relying on the model ILoNDF. This provides a new way of looking at user's need in terms of specificity, exhaustivity and contradictory profile-contributing criteria. These criteria go on to estimate the relative importance the user might attach to precision and recall. The filtering threshold can then be adjusted taking into account this knowledge about user's need. The second direction, complementary to the first one, concerns the refinement of ILoNDF's functionality in order to confer it the capacity of tracking drifting user's need over time. Finally, we consider the generalization of our previous work to the case where streaming data can be divided into multiple classes.NANCY1-Bib. numérique (543959902) / SudocSudocFranceF

Incremental Data-driven Learning of a Novelty Detection Model for One-Class Classification with Application to High-Dimensional Noisy Data

International audienceMost conventional learning algorithms require both positive and negative training data for achieving accurate classification results. However, the problem of learning classifiers from only positive data arises in many applications where negative data are too costly, difficult to obtain, or not available at all. This paper describes a new machine learning approach, called ILoNDF (Incremental data-driven Learning of Novelty Detector Filter). The approach is inspired by novelty detection theory and its learning method, which typically requires only examples from one class to learn a model. One advantage of ILoNDF is the ability of its generative learning to capture the intrinsic characteristics of the training data by continuously integrating the information relating to the relative frequencies of the features of training data and their co-occurrence dependencies. This makes ILoNDF rather stable and less sensitive to noisy features which may be present in the representation of the positive data. In addition, ILoNDF does not require extensive computational resources since it operates on-line without repeated training, and no parameters need to be tuned. In this study we mainly focus on the robustness of ILoNDF in dealing with high-dimensional noisy data and we investigate the variation of its performance depending on the amount of data available for training. To make our study comparable to previous studies, we investigate four common methods: PCA residuals, Hotelling's T2 test, an auto-associative neural network, and a one-class version of the SVM classifier (lately a favored method for one-class classification). Experiments are conducted on two real-world text corpora: Reuters and WebKB. Results show that ILoNDF tends to be more robust, is less affected by initial settings, and consistently outperforms the other methods

Pattern Separation in the Hippocampus: Distinct Circuits under Different Conditions

International audiencePattern separation is a fundamental hippocampal process thought to be critical for distinguishing similar episodic memories, and has long been recognized as a natural function of the dentate gyrus (DG) supporting autoassociative learning in CA3. Understanding how neural circuits within the DG-CA3 network mediate this process has received much interest, yet the exact mechanisms behind remain elusive. Here we argue for the case that sparse coding is necessary but not sufficient to ensure efficient separation and, alternatively, propose a possible interaction of distinct circuits which, nevertheless, act in synergy to produce a unitary function of pattern separation. The proposed circuits involve different functional granule-cell populations, a primary population mediates sparsification and provides recurrent excitation to the other populations which are related to additional pattern separation mechanisms with higher degrees of robustness against interference in CA3. A variety of top-down and bottom-up factors, such as motivation, emotion, and pattern similarity, controls the selection of circuitry depending on circumstances. According to this framework, a computational model is implemented and tested against model variants in a series of numerical simulations and biological experiments. The results demonstrate that the model combines fast learning, robust pattern separation and high storage capacity. It also accounts for the controversy around the involvement of the DG during memory recall, explains other puzzling findings, and makes predictions that can inform future investigations

Towards a Synthetic Analysis of User's Information Need for More Effective Personalized Filtering Services

International audienceThe consideration of underlying analysis of user's information need is a key requirement in an intelligent filtering environment. However, the majority of current approaches to filtering are relevance-oriented, rather than user-oriented. This is partly because they are issued from fields that have somewhat different perspectives from that of information filtering, but also because of the difficulty of understanding and measuring user's motivations and the way in which the user expects the system to respond. This paper presents an original approach to information analysis and filtering inspired by the novelty detection theory. As well as being able to accurately learn user's information need, the approach has an analytical capacity for better understanding user's need. It provides a new way of looking at user's need in terms of precise, broad, and contradictory profile-contributing criteria. These criteria go on to estimate the relative importance the user might attach to precision and recall. The filtering threshold is then adjusted taking into account this knowledge about user's need. Experimental results on the standard Reuters-21578 collection prove the effectiveness of the approach and confirm the potential usefulness of adapting the filtering results according to the knowledge acquired about user's need

An Innovative Approach to Intelligent Information Filtering

International audienceInformation filtering is one of the most useful and challenging tasks for effective information access. It is concerned with dynamically adapting the distribution of information where both evolving user's interests and new incoming information are taken into account. In this paper, we present an innovative approach to text filtering based on the novelty detection principle. This approach relies on a specific learning model which allows both accurate online learning of user's profile and evaluation of the coherency of user's behaviour during his interaction with the system. We empirically analyse our approach and present experimental results on the Reuters-21578 benchmark. The obtained results bring out a significant enhancement of performance as compared to the widely used Rocchio's learning algorithm