Language Grounding through Social Interactions and Curiosity-Driven Multi-Goal Learning

International audienceAutonomous reinforcement learning agents, like children, do not have access to predefined goals and reward functions. They must discover potential goals, learn their own reward functions and engage in their own learning trajectory. Children, however, benefit from exposure to language, helping to organize and mediate their thought. We propose LE2 (Language Enhanced Exploration), a learning algorithm leveraging intrinsic motivations and natural language (NL) interactions with a descriptive social partner (SP). Using NL descriptions from the SP, it can learn an NL-conditioned reward function to formulate goals for intrinsically motivated goal exploration and learn a goal-conditioned policy. By exploring, collecting descriptions from the SP and jointly learning the reward function and the policy, the agent grounds NL descriptions into real behavioral goals. From simple goals discovered early to more complex goals discovered by experimenting on simpler ones, our agent autonomously builds its own behavioral repertoire. This naturally occurring curriculum is supplemented by an active learning curriculum resulting from the agent's intrinsic motivations. Experiments are presented with a simulated robotic arm that interacts with several objects including tools

Selecting biomedical data sources according to user preferences

Motivation: Biologists are now faced with the problem of integrating information from multiple heterogeneous public sources with their own experimental data contained in individual sources. The selection of the sources to be considered is thus critically important. Results: Our aim is to support biologists by developing a module based on an algorithm that presents a selection of sources relevant to their query and matched to their own preferences. We approached this task by investigating the characteristics of biomedical data and introducing several preference criteria useful for bioinformaticians. This work was carried out in the framework of a project which aims to develop an integrative platform for the multiple parametric analysis of cancer. We illustrate our study through an elementary biomedical query occurring in a CGH analysis scenario

Language Grounding through Social Interactions and Curiosity-Driven Multi-Goal Learning

International audienceAutonomous reinforcement learning agents, like children, do not have access to predefined goals and reward functions. They must discover potential goals, learn their own reward functions and engage in their own learning trajectory. Children, however, benefit from exposure to language, helping to organize and mediate their thought. We propose LE2 (Language Enhanced Exploration), a learning algorithm leveraging intrinsic motivations and natural language (NL) interactions with a descriptive social partner (SP). Using NL descriptions from the SP, it can learn an NL-conditioned reward function to formulate goals for intrinsically motivated goal exploration and learn a goal-conditioned policy. By exploring, collecting descriptions from the SP and jointly learning the reward function and the policy, the agent grounds NL descriptions into real behavioral goals. From simple goals discovered early to more complex goals discovered by experimenting on simpler ones, our agent autonomously builds its own behavioral repertoire. This naturally occurring curriculum is supplemented by an active learning curriculum resulting from the agent's intrinsic motivations. Experiments are presented with a simulated robotic arm that interacts with several objects including tools

Bioinformatics for precision medicine in oncology: principles and application to the SHIVA clinical trial

Precision medicine (PM) requires the delivery of individually adapted medical care based on the genetic characteristics of each patient and his/her tumor. The last decade witnessed the development of high-throughput technologies such as microarrays and next-generation sequencing which paved the way to PM in the field of oncology. While the cost of these technologies decreases, we are facing an exponential increase in the amount of data produced. Our ability to use this information in daily practice relies strongly on the availability of an efficient bioinformatics system that assists in the translation of knowledge from the bench towards molecular targeting and diagnosis. Clinical trials and routine diagnoses constitute different approaches, both requiring a strong bioinformatics environment capable of (i) warranting the integration and the traceability of data, (ii) ensuring the correct processing and analyses of genomic data, and (iii) applying well-defined and reproducible procedures for workflow management and decision-making. To address the issues, a seamless information system was developed at Institut Curie which facilitates the data integration and tracks in real-time the processing of individual samples. Moreover, computational pipelines were developed to identify reliably genomic alterations and mutations from the molecular profiles of each patient. After a rigorous quality control, a meaningful report is delivered to the clinicians and biologists for the therapeutic decision. The complete bioinformatics environment and the key points of its implementation are presented in the context of the SHIVA clinical trial, a multicentric randomized phase II trial comparing targeted therapy based on tumor molecular profiling versus conventional therapy in patients with refractory cancer. The numerous challenges faced in practice during the setting up and the conduct of this trial are discussed as an illustration of PM application

Prevalence of the metabolic syndrome in Luxembourg according to the Joint Interim Statement definition estimated from the ORISCAV-LUX study

ABSTRACT: BACKGROUND: The prevalence of the metabolic syndrome (MS) has been determined in many countries worldwide but never in Luxembourg. This research aimed to 1) establish the gender- and age-specific prevalence of MS and its components in the general adult population of Luxembourg, according to the most recent Joint Interim Statement (JIS) definition, by using both the high and low cut-off points to define abdominal obesity, and 2) compare and assess the degree of agreement with the Revised National Cholesterol Education Programme-Adult Treatment Panel III (R-ATPIII) and the International Diabetes Federation (IDF) definitions. METHODS: A representative stratified random sample of 1349 European subjects, aged 18-69 years, participated to ORISCAV-LUX survey. Logistic regression and odds ratios (OR) were used to study MS prevalence with respect to gender and age. The Framingham risk score (FRS) to predict the 10-year coronary heart disease (CHD) risk was calculated to compare the proportion of MS cases below or above 20%, according to both high and low waist circumference (WC) thresholds. Cohen's kappa coefficient (kappa) was utilized to measure the degree of agreement between MS definitions. RESULTS: The prevalence of the MS defined by the JIS was 28.0% and 24.7% when using the low (94/80) and the high (102/88) WC cut-off points, respectively. The prevalence was significantly higher in men than in women (OR = 2.6 and 2.3 for the low and high WC thresholds), as were all components of the MS except abdominal obesity measured by both thresholds. It also increased with age (OR values in age categories ranging from 2.7 to 28 when compared to the younger subjects for low WC and from 3.3 to 31 for the high WC cut-offs). The 10-year predicted risk of CHD by FRS did not depend on the threshold used. Globally, excellent agreement was observed between the three definitions of MS (kappa= 0.89), in particular between JIS and IDF (kappa = 0.93). Agreement was significantly higher in women than in men, and differed between age groups. CONCLUSION: Regardless of the definition used, the adult population of Luxembourg reveals a high MS prevalence. Our findings contribute to build evidence regarding the definitive construct of the MS, to help selecting the waist circumference thresholds for Europid populations, and to support the need to revise the guidelines for abdominal obesity levels

First nationwide survey on cardiovascular risk factors in Grand-Duchy of Luxembourg (ORISCAV-LUX)

BACKGROUND: The ORISCAV-LUX study is the first baseline survey of an on-going cardiovascular health monitoring programme in Grand-Duchy of Luxembourg. The main objectives of the present manuscript were 1) to describe the study design and conduct, and 2) to present the salient outcomes of the study, in particular the prevalence of the potentially modifiable and treatable cardiovascular disease risk factors in the adult population residing in Luxembourg. METHOD: ORISCAV-LUX is a cross-sectional study based on a random sample of 4496 subjects, stratified by gender, age categories and district, drawn from the national insurance registry of 18-69 years aged Luxembourg residents, assuming a response rate of 30% and a proportion of 5% of institutionalized subjects in each stratum. The cardiovascular health status was assessed by means of a self-administered questionnaire, clinical and anthropometric measures, as well as by blood, urine and hair examinations. The potentially modifiable and treatable risk factors studied included smoking, hypertension, dyslipidemia, diabetes mellitus, and obesity. Both univariate and multivariate statistical analyses used weighted methods to account for the stratified sampling scheme. RESULTS: A total of 1432 subjects took part in the survey, yielding a participation rate of 32.2%. This figure is higher than the minimal sample size of 1285 subjects as estimated by power calculation. The most predominant cardiovascular risk factors were dyslipidemia (69.9%), hypertension (34.5%), smoking (22.3%), and obesity (20.9%), while diabetes amounted 4.4%. All prevalence rates increased with age (except smoking) with marked gender differences (except diabetes). There was a significant difference in the prevalence of hypertension and of lipid disorders by geographic region of birth. The proportion of subjects cumulating two or more cardiovascular risk factors increased remarkably with age and was more predominant in men than in women (P<0.0001). Only 14.7% of men and 23.1% of women were free of any cardiovascular risk factor. High prevalence of non-treated CVRF, notably for hypertension and dyslipidemia, were observed in the study population. CONCLUSION: The population-based ORISCAV-LUX survey revealed a high prevalence of potentially modifiable and treatable cardiovascular risk factors among apparently healthy subjects; significant gender and age-specific differences were seen not only for single but also for combined risk factors. From a public health perspective, these preliminary findings stress the urgent need for early routine health examinations, preventive interventions and lifestyle behavioural changes, even in young asymptomatic adults, to decrease cardiovascular morbidity and mortality in Luxembourg

Comparison of participants and non-participants to the ORISCAV-LUX population-based study on cardiovascular risk factors in Luxembourg

BACKGROUND: Poor response is a major concern in public health surveys. In a population-based ORISCAV-LUX study carried out in Grand-Duchy of Luxembourg to assess the cardiovascular risk factors, the non-response rate was not negligible. The aims of the present work were: 1) to investigate the representativeness of study sample to the general population, and 2) to compare the known demographic and cardiovascular health-related profiles of participants and non-participants. METHODS: For sample representativeness, the participants were compared to the source population according to stratification criteria (age, sex and district of residence). Based on complementary information from the "medical administrative database", further analysis was carried out to assess whether the health status affected the response rate. Several demographic and morbidity indicators were used in the univariate comparison between participants and non-participants. RESULTS: Among the 4452 potentially eligible subjects contacted for the study, there were finally 1432 (32.2%) participants. Compared to the source population, no differences were found for gender and district distribution. By contrast, the youngest age group was under-represented while adults and elderly were over-represented in the sample, for both genders. Globally, the investigated clinical profile of the non-participants was similar to that of participants. Hospital admission and cardiovascular health-related medical measures were comparable in both groups even after controlling for age. The participation rate was lower in Portuguese residents as compared to Luxembourgish (OR = 0.58, 95% CI: 0.48-0.69). It was also significantly associated with the professional status (P < 0.0001). Subjects from the working class were less receptive to the study than those from other professional categories. CONCLUSION: The 32.2% participation rate obtained in the ORISCAV-LUX survey represents the realistic achievable rate for this type of multiple-stage, nationwide, population-based surveys. It corresponds to the expected rate upon which the sample size was calculated. Given the absence of discriminating health profiles between participants and non-participants, it can be concluded that the response rate does not invalidate the results and allows generalizing the findings for the population

Langage et Apprentissage en Interaction pour des Assistants Numériques Autonomes - Une Approche Développementale

The rapid development of digital assistants (DA) opens the way to new modes of interaction. Some DA allows users to personalise the way they respond to queries, in particular by teaching them new procedures. This work proposes to use machine learning methods to enrich the linguistic and procedural generalisation capabilities of these systems. The challenge is to reconcile rapid learning skills, necessary for a smooth user experience, with a sufficiently large generalisation capacity. Though this is a natural human ability, it remains out-of-reach for artificial systems and this leads us to approach these issues from the perspective of developmental Artificial Intelligence. This work is thus inspired by the cognitive processes at work in children during language learning.First, we propose a language processing module, which relies on semantic comparison methods to interpret the user’s natural language requests. The variability of a user speech is indeed one of the main difficulties of these learning assistants. We provide them with a generalisation tool to continuously adapt to the user language. Another challenge for these learning agents is their ability to transfer their knowledge to new objects and contexts. We propose a series of architectures for Deep Reinforcement Learning agents that learn to perform tasks expressed in natural language in various environments. By exploiting language as an abstraction tool to represent tasks, we show that in structured environment, these agents are able to transfer their skills to new objects.Finally, we develop a use case in a home automation environment. We propose a learning assistant that integrates the systems mentioned above.L’essor des assistants numériques ouvre la voie à de nouveaux modes d’interaction. Les assistants apprenant par interaction offrent ainsi la possibilité aux utilisateurs de personnaliser la manière dont ils répondent aux requêtes, notamment en leur enseignant de nouvelles procédures. Ces travaux proposent de s’appuyer sur des méthodes d’apprentissage automatique pour enrichir les capacités de généralisation linguistique et procédurale de ces systèmes. L’enjeu consiste à concilier des facultés d’apprentissage rapide, nécessaire à une expérience utilisateur fluide, avec une capacité de généralisation suffisamment large. Naturels pour les humains, ces qualités restent encore hors de portée pour les systèmes artificiels. Cela nous conduit à approcher ces problématiques sous l’angle de l’Intelligence Artificielle développementale. Ces travaux sont ainsi inspirés par les processus cognitifs à l’œuvre chez l’enfant lors de l’apprentissage du langage.On propose tout d’abord un module de traitement du langage, qui s’appuie sur des méthodes de comparaison sémantique pour interpréter les requêtes en langage naturel de l’utilisateur. La variabilité avec laquelle un locuteur s’exprime constitue en effet l’une des difficultés centrales de ces assistants apprenants. Notre module leur permet de disposer d’un outil de généralisation sur le langage qui s’adapte en continu à l’utilisateur. Un autre enjeu pour ces agents apprenants est leur capacité à transférer leur connaissance à de nouveaux objets et contextes. On propose alors une série d’architectures d’agents de Deep Reinforcement Learning apprenant à exécuter des tâches exprimées en langage naturel dans des environnements variés. En exploitant le langage comme outil d’abstraction pour représenter les tâches, on montre que lorsque l’environnement est suffisamment structuré, ces agents sont capables de transférer les compétences acquises sur certains objets à de nouveaux.On développe enfin un cas d’usage dans un environnement domotique. On propose un assistant apprenant qui intègre les systèmes mentionnés précédemmen