Système intelligent pour le suivi et l’optimisation de l’état cognitif

Les émotions des êtres humains changent régulièrement et parfois de manière brusque entrainant un changement de l’état mental c’est-à-dire de l’aptitude cérébrale à fonctionner normalement. Il en résulte une capacité cognitive (ou état cognitif) de l’individu à pouvoir raisonner, accéder à la mémoire, ou effectuer des déductions, variable selon l’état mental. Ceci affecte, en conséquence, les performances des utilisateurs qui varient en fonction de leurs état cognitifs. Cette thèse vise à optimiser l’état cognitif d’un utilisateur lors de ses interactions avec un environnement virtuel. Comme cet état dépend des émotions, l’optimisation de l’état cognitif peut être réalisée à travers l’optimisation des émotions et en particulier la réduction des émotions négatives. Une première partie concerne les moyens de mesurer en temps réel (par un Module de mesures) l’état émotionnel et mental d’un utilisateur lors de ses interactions avec un environnement virtuel. Nous avons réalisé pour cela quatre études expérimentales avec quatre environnements différents. Nous avons montré que ces mesures peuvent être réalisées en utilisant différents capteurs physiologiques. Nous avons aussi montré qu’il est possible de prédire la tendance de l’excitation (un état mental) à partir d’un traceur de regard. Dans une deuxième partie, nous présentons l’Agent Neural qui modifie les environnements virtuels afin de provoquer une modification de l’état émotionnel d’un utilisateur pour améliorer son état cognitif. Nous avons réalisé quatre études expérimentales avec quatre environnements virtuels, où l’Agent Neural intervient dans ces environnements afin de changer l’état émotionnel de l’utilisateur. Nous avons montré que l’agent est capable d’intervenir dans plusieurs types d’environnements et de modifier les émotions de l’utilisateur. Dans une troisième partie, présentons l’Agent Limbique, qui personnalise et améliore les adaptations faites par l’Agent Neural à travers l’observation et l’apprentissage des impacts des changements des environnements virtuels et des réactions émotionnelles des utilisateurs. Nous avons montré que cet agent est capable d’analyser les interventions de l’Agent Neural et de les modifier. Nous avons montré aussi que l’Agent Limbique est capable de générer une nouvelle règle d’intervention et de prédire son impact sur l’utilisateur. La combinaison du Module de mesures, de l’Agent Neural, et de l’Agent Limbique, nous a permis de créer un système de contrôle cognitif intelligent que nous avons appelé Système Limbique Digital.The human’s emotions change regularly and sometimes suddenly leading to changes in their mental state which is the brain’s ability to function normally. This mental state’s changes affect the users’ cognitive ability (or cognitive state) to reason, access memory, or make inferences, which varies depending on the mental state. Consequently, this affects the users’ performances which varies according to their cognitive states. This thesis aims to optimize the users’ cognitive state during their interactions with a virtual environment. Since this state depends on emotions, optimization of cognitive state can be achieved through the optimization of emotions and in particular the reduction of negative emotions. In a first part, we present the means of measuring in real time (using a Measuring module) the users’ emotional and mental state during their interactions with a virtual environment. We performed four experimental studies with four different environments. We have shown that these measurements can be performed using different physiological sensors. We have also shown that it is possible to predict the tendency of excitement (a mental state) using an eye tracker. In a second part, we present the Neural Agent which modifies virtual environments to provoke a modification on the users’ emotional state in order to improve their cognitive state. We performed four experimental studies with four virtual environments, in which the Neural Agent intervenes in these environments to change the users’ emotional state. We have shown that the agent is able to intervene in several types of environments and able to modify the users’ emotions. In a third part, we present the Limbic Agent, which personalizes and improves the adaptations performed by the Neural Agent through the observation and the learning from the virtual environments changes’ impacts and the users’ emotional reactions. We have shown that this agent is able to analyze the Neural Agent’s interventions and able to modify them. We have also shown that the Limbic Agent is able to generate a new intervention rule and predict its impact on the user. The combination of the Measuring Module, the Neural Agent, and the Limbic Agent, allowed us to create an intelligent cognitive control system that we called the Digital Limbic System

Gaps : geometry-aware problem solver

Geometry problem solving presents a formidable challenge within the NLP community. Existing approaches often rely on models designed for solving math word problems, neglecting the unique characteristics of geometry math problems. Additionally, the current research predominantly focuses on geometry calculation problems, while overlooking other essential aspects like proving. In this study, we address these limitations by proposing the Geometry-Aware Problem Solver (GAPS) model. GAPS is specifically designed to generate solution programs for geometry math problems of various types with the help of its unique problem-type classifier. To achieve this, GAPS treats the solution program as a composition of operators and operands, segregating their generation processes. Furthermore, we introduce the geometry elements enhancement method, which enhances the ability of GAPS to recognize geometry elements accurately. By leveraging these improvements, GAPS showcases remarkable performance in resolving geometry math problems. Our experiments conducted on the UniGeo dataset demonstrate the superiority of GAPS over the state-of-the-art model, Geoformer. Specifically, GAPS achieves an accuracy improvement of more than 5.3% for calculation tasks and an impressive 41.1% for proving tasks. Notably, GAPS achieves an impressive accuracy of 97.5% on proving problems, representing a significant advancement in solving geometry proving tasks

Static Generation of UML Sequence Diagrams

UML sequence diagrams are visual representations of object interactions in a system and can provide valuable information for program comprehension, debugging, maintenance, and software archeology. Sequence diagrams generated from legacy code are independent of existing documentation that may have eroded. We present a framework for static generation of UML sequence diagrams from object-oriented source code. The framework provides a query refinement system to guide the user to interesting interactions in the source code. Our technique involves constructing a hypergraph representation of the source code, traversing the hypergraph with respect to a user-defined query, and generating the corresponding set of sequence diagrams. We implemented our framework as a tool, StaticGen (supporting software: StaticGen), analyzing a corpus of 30 Android applications. We provide experimental results demonstrating the efficacy of our technique (originally appeared in the Proceedings of Fundamental Approaches to Software Engineering—20th International Conference, FASE 2017, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2017, Uppsala, Sweden, April 22–29, 2017)

Advances in statistical script learning

When humans encode information into natural language, they do so with the clear assumption that the reader will be able to seamlessly make inferences based on world knowledge. For example, given the sentence ``Mrs. Dalloway said she would buy the flowers herself,'' one can make a number of probable inferences based on event co-occurrences: she bought flowers, she went to a store, she took the flowers home, and so on. Observing this, it is clear that many different useful natural language end-tasks could benefit from models of events as they typically co-occur (so-called script models). Robust question-answering systems must be able to infer highly-probable implicit events from what is explicitly stated in a text, as must robust information-extraction systems that map from unstructured text to formal assertions about relations expressed in the text. Coreference resolution systems, semantic role labeling, and even syntactic parsing systems could, in principle, benefit from event co-occurrence models. To this end, we present a number of contributions related to statistical event co-occurrence models. First, we investigate a method of incorporating multiple entities into events in a count-based co-occurrence model. We find that modeling multiple entities interacting across events allows for improved empirical performance on the task of modeling sequences of events in documents. Second, we give a method of applying Recurrent Neural Network sequence models to the task of predicting held-out predicate-argument structures from documents. This model allows us to easily incorporate entity noun information, and can allow for more complex, higher-arity events than a count-based co-occurrence model. We find the neural model improves performance considerably over the count-based co-occurrence model. Third, we investigate the performance of a sequence-to-sequence encoder-decoder neural model on the task of predicting held-out predicate-argument events from text. This model does not explicitly model any external syntactic information, and does not require a parser. We find the text-level model to be competitive in predictive performance with an event level model directly mediated by an external syntactic analysis. Finally, motivated by this result, we investigate incorporating features derived from these models into a baseline noun coreference resolution system. We find that, while our additional features do not appreciably improve top-level performance, we can nonetheless provide empirical improvement on a number of restricted classes of difficult coreference decisions.Computer Science

A study of virtual reality-mediated affective state and cognitive decline in Alzheimer’s disease

NeuroscienceLa démence de type d’Alzheimer est la plus commune des démences. Elle entraîne un déclin dans les capacités cognitives et fonctionnelles, se traduisant dans des difficultés au niveau de la prise de décision, de l’accomplissement de tâches quotidiennes, de la communication ainsi qu’au niveau de la mémoire et de l’attention. On remarque également une diminution de l’état émotionnel et une apathie chez ces patients. Ce mémoire explore une nouvelle approche pour atténuer les effets psychologiques et cognitifs de la maladie. Les recherches effectuées dans ce mémoire explorent les impacts cognitifs et les effets sur le bien-être d'une intervention utilisant la réalité virtuelle sur les personnes souffrant de déclin cognitif subjectif. Deux environnements virtuels ont été testés : le premier étant un environnement dans lequel le participant voyage en train à travers différents climats, et le second étant un environnement de musicothérapie qui s’adapte en fonction de la réponse émotionnelle du participant. Pour mesurer les impacts sur l'état affectif, des lectures électroencéphalographiques ont été prises et analysées afin de déduire l'émotion ressentie par le participant avant, pendant et après l'expérience. Les résultats montrent une amélioration générale de l'état émotionnel pour les deux environnements. Quant à la mesure des effets sur les fonctions cognitives, des tâches d'attention et de mémoire ont été effectuées par les participants avant et après l'immersion. Les résultats montrent une légère amélioration des capacités d'attention et une meilleure amélioration de la mémoire. Nous approprions cet écart dans l'expérience de musicothérapie à l'activation musicale d'un réseau de structures cérébrales impliquées dans les expériences agréables : le circuit de récompense. Nous proposons que la musique facilite la rétention de la mémoire chez les personnes souffrant de démence. En effet, les résultats de l’amélioration des fonctions cognitives pour les deux expériences précédentes dépendent fortement de la précision de l'outil de mesure cognitive utilisé pour évaluer les performances d'attention et de mémoire avant et après l'intervention. Pour assurer cette précision, ce mémoire présente un outil de mesure des performances cognitives basé sur des tâches cognitives qui ont montré à plusieurs reprises leur fiabilité. Cet outil d’adresse aux personnes atteintes de la maladie d'Alzheimer pré-clinique et diagnostiquée.Alzheimer’s disease is an irreversible disease which causes progressive memory loss and cognitive decline, eventually leading to severe inability to perform basic day-to-day tasks. The urgency to find an effective cure to the disease is crucial, as the medical and economical spin-offs could be disastrous. The present thesis explores a novel approach to help attenuate the psychological and cognitive effects of the disease. The research carried out for this thesis explored cognitive effects and impacts on overall well-being of a virtual reality intervention on people suffering from subjective cognitive decline. Two virtual environments were tested: the first being an environment in which the participant travels through different climates by train, and the second being a music therapy environment modified as a function of emotional response. To measure the effects on affective state, electroencephalography readings were taken and analyzed to infer the emotion felt by the participant before, during the experiment. Results show a general improvement in emotional state. To measure the effects of the environments on cognitive functions, attention and memory tasks were carried out by the participants before and after the immersion. Results show a small improvement in attention skills and a more substantial improvement in memory skills. We appropriate this discrepancy in the music therapy experiment to the musical activation of a network of brain structures involved in rewarding and pleasurable experiences. We propose that music could facilitate memory retention in people sufferance for dementia. Importantly, the results of the previous experiments rely heavily on the accuracy of the cognitive measurement tool used to evaluate attention and memory performances before and after the intervention. To provide this accuracy, this thesis presents a cognitive performance measurement tool based on cognitive tasks which have repeatedly shown to output reliable results. This tool is created to serve for people with pre-clinical Alzheimer’s disease and diagnosed Alzheimer’s disease. Additionally, this tool is designed in such a way as to minimize the effects of repetition as well as varying levels of education and language. This thesis presents a novel and promising research in the realms of computer sciences and health care

Managing Uncertainty and Vagueness in Semantic Web

Ο Σημασιολογικός Ιστός στοχεύει στην διεκπεραίωση εργασιών σε υπολογιστικά συστήματα χωρίς την ανθρώπινη παρέμβαση. Προκειμένου να επιτευχθεί ο στόχος αυτός, εισάγεται η έννοια της πληροφορίας που είναι επεξεργάσιμη από μηχανές. Στα περισσότερα προβλήματα, η έννοια της πληροφορίας είναι συνυφασμένη με την έννοια της αβεβαιότητας και της ασάφειας. Και οι δύο έννοιες περιγράφονται με την κοινή ονομασία ατελής πληροφορία. Δεδομένου ότι ο Σημασιολογικός Ιστός απαρτίζεται από ένα σύνολο τεχνολογιών και των θεωριών που τις διέπουν, οποιαδήποτε μέθοδος αναπαράστασης θα πρέπει να βρίσκεται σε συμφωνία με άλλες υπάρχουσες. Συγκεκριμένα, το θεωρητικό πλαίσιο πρέπει να εντάσσεται ομαλά στη θεωρία που εφαρμόζεται στο Σημασιολογικό Ιστό. Η δε υλοποίησή του, ιδανικό είναι, να υποστηριχθεί με χρήση μεθόδων του Σημασιολογικού Ιστού, στις οποίες κυριαρχεί εκείνη των οντολογιών. Στη διατριβή μας, ορίσαμε μία μέθοδο αναπαράστασης της αβεβαιότητας και της ασάφειας μέσω ενός ενιαίου πλαισίου. Το μοντέλο Dempster-Shafer χρησιμοποιήθηκε για την αναπαράσταση της αβεβαιότητας και το μοντέλο Ασαφούς Λογικής και Ασαφών Συνόλων για την αναπαράσταση της ασάφειας. Για το λόγο αυτό, ορίσαμε το θεωρητικό πλαίσιο, στοχεύοντας σε ένα συνδυασμό ALC Λογικών Περιγραφών (Description Logics) με το μοντέλο Dempster-Shafer. Κατά τη διάρκεια της έρευνάς μας υλοποιήσαμε μεταοντολογίες για την αναπαράσταση της αβεβαιότητας και της ασάφειας και στη συνέχεια μελετήσαμε την συμπεριφορά τους σε πραγματικές εφαρμογές.Semantic Web has been designed for processing tasks without human intervention. In this context, the term machine processable information has been introduced. In most Semantic Web tasks, we come across information incompleteness issues, aka uncertainty and vagueness. For this reason, a method that represents uncertainty and vagueness under a common framework has to be defined. Semantic Web technologies are defined through a Semantic Web Stack and are based on a clear formal foundation. Therefore, any representation scheme should be aligned with these technologies and be formally defined. As the concept of ontologies is significant in the Semantic Web for representing knowledge, any framework is desirable to be built upon it. In our work, we have defined an approach for representing uncertainty and vagueness under a common framework. Uncertainty is represented through Dempster-Shafer model, whereas vagueness has been represented through Fuzzy Logic and Fuzzy Sets. For this reason, we have defined our theoretical framework, aimed at a combination of the classical crisp DL ALC with a Dempster-Shafer module. As a next step, we added fuzziness to this model. Throughout our work, we have implemented metaontologies in order to represent uncertain and vague concepts and, next, we have tested our methodology in real-world applications

Exploring attributes, sequences, and time in Recommender Systems: From classical to Point-of-Interest recommendation

Tesis Doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingenieria Informática. Fecha de lectura: 08-07-2021Since the emergence of the Internet and the spread of digital communications throughout the world, the amount of data stored on the Web has been growing exponentially. In this new digital era, a large number of companies have emerged with the purpose of ltering the information available on the web and provide users with interesting items. The algorithms and models used to recommend these items are called Recommender Systems. These systems are applied to a large number of domains, from music, books, or movies to dating or Point-of-Interest (POI), which is an increasingly popular domain where users receive recommendations of di erent places when they arrive to a city. In this thesis, we focus on exploiting the use of contextual information, especially temporal and sequential data, and apply it in novel ways in both traditional and Point-of-Interest recommendation. We believe that this type of information can be used not only for creating new recommendation models but also for developing new metrics for analyzing the quality of these recommendations. In one of our rst contributions we propose di erent metrics, some of them derived from previously existing frameworks, using this contextual information. Besides, we also propose an intuitive algorithm that is able to provide recommendations to a target user by exploiting the last common interactions with other similar users of the system. At the same time, we conduct a comprehensive review of the algorithms that have been proposed in the area of POI recommendation between 2011 and 2019, identifying the common characteristics and methodologies used. Once this classi cation of the algorithms proposed to date is completed, we design a mechanism to recommend complete routes (not only independent POIs) to users, making use of reranking techniques. In addition, due to the great di culty of making recommendations in the POI domain, we propose the use of data aggregation techniques to use information from di erent cities to generate POI recommendations in a given target city. In the experimental work we present our approaches on di erent datasets belonging to both classical and POI recommendation. The results obtained in these experiments con rm the usefulness of our recommendation proposals, in terms of ranking accuracy and other dimensions like novelty, diversity, and coverage, and the appropriateness of our metrics for analyzing temporal information and biases in the recommendations producedDesde la aparici on de Internet y la difusi on de las redes de comunicaciones en todo el mundo, la cantidad de datos almacenados en la red ha crecido exponencialmente. En esta nueva era digital, han surgido un gran n umero de empresas con el objetivo de ltrar la informaci on disponible en la red y ofrecer a los usuarios art culos interesantes. Los algoritmos y modelos utilizados para recomendar estos art culos reciben el nombre de Sistemas de Recomendaci on. Estos sistemas se aplican a un gran n umero de dominios, desde m usica, libros o pel culas hasta las citas o los Puntos de Inter es (POIs, en ingl es), un dominio cada vez m as popular en el que los usuarios reciben recomendaciones de diferentes lugares cuando llegan a una ciudad. En esta tesis, nos centramos en explotar el uso de la informaci on contextual, especialmente los datos temporales y secuenciales, y aplicarla de forma novedosa tanto en la recomendaci on cl asica como en la recomendaci on de POIs. Creemos que este tipo de informaci on puede utilizarse no s olo para crear nuevos modelos de recomendaci on, sino tambi en para desarrollar nuevas m etricas para analizar la calidad de estas recomendaciones. En una de nuestras primeras contribuciones proponemos diferentes m etricas, algunas derivadas de formulaciones previamente existentes, utilizando esta informaci on contextual. Adem as, proponemos un algoritmo intuitivo que es capaz de proporcionar recomendaciones a un usuario objetivo explotando las ultimas interacciones comunes con otros usuarios similares del sistema. Al mismo tiempo, realizamos una revisi on exhaustiva de los algoritmos que se han propuesto en el a mbito de la recomendaci o n de POIs entre 2011 y 2019, identi cando las caracter sticas comunes y las metodolog as utilizadas. Una vez realizada esta clasi caci on de los algoritmos propuestos hasta la fecha, dise~namos un mecanismo para recomendar rutas completas (no s olo POIs independientes) a los usuarios, haciendo uso de t ecnicas de reranking. Adem as, debido a la gran di cultad de realizar recomendaciones en el ambito de los POIs, proponemos el uso de t ecnicas de agregaci on de datos para utilizar la informaci on de diferentes ciudades y generar recomendaciones de POIs en una determinada ciudad objetivo. En el trabajo experimental presentamos nuestros m etodos en diferentes conjuntos de datos tanto de recomendaci on cl asica como de POIs. Los resultados obtenidos en estos experimentos con rman la utilidad de nuestras propuestas de recomendaci on en t erminos de precisi on de ranking y de otras dimensiones como la novedad, la diversidad y la cobertura, y c omo de apropiadas son nuestras m etricas para analizar la informaci on temporal y los sesgos en las recomendaciones producida

Supervised Training on Synthetic Languages: A Novel Framework for Unsupervised Parsing

This thesis focuses on unsupervised dependency parsing—parsing sentences of a language into dependency trees without accessing the training data of that language. Different from most prior work that uses unsupervised learning to estimate the parsing parameters, we estimate the parameters by supervised training on synthetic languages. Our parsing framework has three major components: Synthetic language generation gives a rich set of training languages by mix-and-match over the real languages; surface-form feature extraction maps an unparsed corpus of a language into a fixed-length vector as the syntactic signature of that language; and, finally, language-agnostic parsing incorporates the syntactic signature during parsing so that the decision on each word token is reliant upon the general syntax of the target language. The fundamental question we are trying to answer is whether some useful information about the syntax of a language could be inferred from its surface-form evidence (unparsed corpus). This is the same question that has been implicitly asked by previous papers on unsupervised parsing, which only assumes an unparsed corpus to be available for the target language. We show that, indeed, useful features of the target language can be extracted automatically from an unparsed corpus, which consists only of gold part-of-speech (POS) sequences. Providing these features to our neural parser enables it to parse sequences like those in the corpus. Strikingly, our system has no supervision in the target language. Rather, it is a multilingual system that is trained end-to-end on a variety of other languages, so it learns a feature extractor that works well. This thesis contains several large-scale experiments requiring hundreds of thousands of CPU-hours. To our knowledge, this is the largest study of unsupervised parsing yet attempted. We show experimentally across multiple languages: (1) Features computed from the unparsed corpus improve parsing accuracy. (2) Including thousands of synthetic languages in the training yields further improvement. (3) Despite being computed from unparsed corpora, our learned task-specific features beat previous works’ interpretable typological features that require parsed corpora or expert categorization of the language