From Phonemes to Sentence Comprehension: A Neurocomputational Model of Sentence Processing for Robots

International audienceThere has been an important progress these last years in speech recognition systems. The word recognition error rate went down with the arrival of deep learning methods. However, if one uses cloud speech API and integrate it inside a robotic architecture, one faces a non negligible number of wrong sentence recognition. Thus speech recognition can not be considered as solved (because many sentences out of their contexts are ambiguous). We believe that contextual solutions (i.e. adaptable and trainable on different HRI applications) have to be found. In this perspective, the way children learn language and how our brains process utterances may help us improve how robots process language. Getting inspiration from language acquisition theories and how the brain processes sentences we previously developed a neuro-inspired model of sentence processing. In this study, we investigate how this model can process different levels of abstractions as input: sequence of phonemes, seq. of words or grammatical constructions. We see that even if the model was only tested on grammatical constructions before, it has better performances with words and phonemes inputs

La linguistique, un modèle à deux faces

L’énaction est dite partager certaines caractéristiques du embodiment (incorporation) et du extended mind (esprit étendu) lesquelles la linguistique devrait profiter. Cependant leurs directions respectives sont opposées. La linguistique devrait assurer un formalisme qui rendait possible la transition entre ces deux tensions opposées: la topologie est justement une telle méthode parce qu’elle permet d’opposer le langage, où les unités sont enchainés, au métalangage, où des éléments dans un paradigme son choisis

Cross-Situational Learning with Reservoir Computing for Language Acquisition Modelling

International audienceUnderstanding the mechanisms enabling children to learn rapidly word-to-meaning mapping through cross-situational learning in uncertain conditions is still a matter of debate. In particular, many models simply look at the word level, and not at the full sentence comprehension level. We present a model of language acquisition, applying cross-situational learning on Recurrent Neural Networks with the Reservoir Computing paradigm. Using the co-occurrences between words and visual perceptions, the model learns to ground a complex sentence, describing a scene involving different objects, into a perceptual representation space. The model processes sentences describing scenes it perceives simultaneously via a simulated vision module: sentences are inputs and simulated vision are target outputs of the RNN. Evaluations of the model show its capacity to extract the semantics of virtually hundred of thousands possible combinations of sentences (based on a context-free grammar); remarkably the model generalises only after a few hundred of partially described scenes via cross-situational learning. Furthermore, it handles polysemous and synonymous words, and deals with complex sentences where word order is crucial for understanding. Finally, further improvements of the model are discussed in order to reach proper reinforced and self-supervised learning schemes, with the goal to enable robots to acquire and ground language by themselves (with no oracle supervision)

A Journey in ESN and LSTM Visualisations on a Language Task

Echo States Networks (ESN) and Long-Short Term Memory networks (LSTM) are two popular architectures of Recurrent Neural Networks (RNN) to solve machine learning task involving sequential data. However, little have been done to compare their performances and their internal mechanisms on a common task. In this work, we trained ESNs and LSTMs on a Cross-Situationnal Learning (CSL) task. This task aims at modelling how infants learn language: they create associations between words and visual stimuli in order to extract meaning from words and sentences. The results are of three kinds: performance comparison, internal dynamics analyses and visualization of latent space. (1) We found that both models were able to successfully learn the task: the LSTM reached the lowest error for the basic corpus, but the ESN was quicker to train. Furthermore, the ESN was able to outperform LSTMs on datasets more challenging without any further tuning needed. (2) We also conducted an analysis of the internal units activations of LSTMs and ESNs. Despite the deep differences between both models (trained or fixed internal weights), we were able to uncover similar inner mechanisms: both put emphasis on the units encoding aspects of the sentence structure. (3) Moreover, we present Recurrent States Space Visualisations (RSSviz), a method to visualize the structure of latent state space of RNNs, based on dimension reduction (using UMAP). This technique enables us to observe a fractal embedding of sequences in the LSTM. RSSviz is also useful for the analysis of ESNs (i) to spot difficult examples and (ii) to generate animated plots showing the evolution of activations across learning stages. Finally, we explore qualitatively how the RSSviz could provide an intuitive visualisation to understand the influence of hyperparameters on the reservoir dynamics prior to ESN training

Studio ed implementazione di un sistema basato sulle reti neurali per il calcolo della percezione attesa di flusso ottico

Nel lavoro di tesi presentato assieme a questo frontespizio si modella ed implementa un'architettura per il controllo della locomozione di un robot attraverso la predizione dei dati sensoriali visivi. Si incomincia con un'analisi dello stato dell'arte. Si analizza poi più dettagliatamente una soluzione al problema basata su di un particolare modello interno anticipativo chiamato EP Scheme. A partire da questa viene presentato un'architettura che basa il suo funzionamento sulla predizione del flusso ottico attraverso una rete neurale ricorrente (ESN). Viene in fine implementato e testato su simulatore il modello e ne vengono analizzati i risultati

Análisis enactivo de las relaciones actanciales desde las diferencias de género. Relaciones entre sintaxis, semántica, cognición y modelos sociales y mentales de roles de género

El propósito de la presente investigación es realizar un análisis lingüístico de textos extraídos de novela rosa española de diferentes épocas (siglo XX y actual) desde la teoría de la enacción. El análisis se articula alrededor de las diferencias de género. Se parte del dominio emocional de las relaciones amorosas entre hombres y mujeres, que son las que caracterizan la novela rosa trabajada. Desde ese punto de partida, el estudio defiende la interrelación entre los roles sociales y mentales de los seres humanos de acuerdo con los roles de género y los papeles semánticos en el lenguaje. También se demuestra la relación de estos aspectos con los patrones actanciales y construccionales. El trabajo constata la existencia de papeles actanciales estáticos y dinámicos que dependen de la diferencia de género. El estudio se plantea hasta qué punto hay una evolución de los hechos lingüísticos en los trabajos de épocas distintas. Este trabajo constituye el proyecto de investigación para la plaza de profesora titular de universidad de su autora, presentado en julio de 2018 en la Universitat de València.The purpose of this research is to carry out a linguistic analysis of texts extracted from Spanish romance novels from different periods (20th century and current) from the theory of enaction. The analysis is articulated around gender differences. It starts from the emotional domain of love relationships between men and women, which are the ones that characterize the rose novel worked. From this starting point, the study defends the interrelation between the social and mental roles of human beings according to gender roles and semantic roles in language. The relationship of these aspects with actantial and constructional patterns is also demonstrated. The work confirms the existence of static and dynamic acting roles that depend on the gender difference. The study considers to what extent there is an evolution of the linguistic facts in the works of different periods. This work constitutes the research project for the position of tenured university professor of its author, presented in July 2018 at the University of Valencia

Humanoid Robots

For many years, the human being has been trying, in all ways, to recreate the complex mechanisms that form the human body. Such task is extremely complicated and the results are not totally satisfactory. However, with increasing technological advances based on theoretical and experimental researches, man gets, in a way, to copy or to imitate some systems of the human body. These researches not only intended to create humanoid robots, great part of them constituting autonomous systems, but also, in some way, to offer a higher knowledge of the systems that form the human body, objectifying possible applications in the technology of rehabilitation of human beings, gathering in a whole studies related not only to Robotics, but also to Biomechanics, Biomimmetics, Cybernetics, among other areas. This book presents a series of researches inspired by this ideal, carried through by various researchers worldwide, looking for to analyze and to discuss diverse subjects related to humanoid robots. The presented contributions explore aspects about robotic hands, learning, language, vision and locomotion