Users’ Reactions Captured by Means of an EEG Headset on Viewing the Presentation of Sustainable Designs Using Verbal Narrative

The aim of this paper is to determine whether consu mers accept new arguments for choosing a product that adapts to future needs. It is also seeks to investigate whether the design of products and their ensuing advertising an d promotion through a sustainable approach by means of verbal narrative ads can gener ate a more positive emotional response in the future users of the product than wi th the application of visual narrative ads. To this end, an experiment was conducted consisting in consumers, with and without experience with the product, watching a promotional video based on verbal narrative, created using the new usage scenarios approach, in which the advantages of a sustainable product are shown. The neuronal respons e of the possible users was then measured by means of the EEG headset. In order to b e able to establish a comparison, the same response was also measured in the same con sumers when they viewed a commercial video based on visual narrative about a product with similar characteristics. The results show, among other conclusions, that vie wing the verbal narrative ad first triggers higher emotional values of excitement, bot h in the short and the long term, as well as frustration. It is also observed that havin g no experience with the product causes higher meditation values. This can be useful to enterprises both in order to design their products in such a way as to orientate them towards consumer concerns, and to design advertisements in such a way as to link consumers emotionally with the produ ct

Integrating EEG and MEG signals to improve motor imagery classification in brain-computer interfaces

We propose a fusion approach that combines features from simultaneously recorded electroencephalographic (EEG) and magnetoencephalographic (MEG) signals to improve classification performances in motor imagery-based brain-computer interfaces (BCIs). We applied our approach to a group of 15 healthy subjects and found a significant classification performance enhancement as compared to standard single-modality approaches in the alpha and beta bands. Taken together, our findings demonstrate the advantage of considering multimodal approaches as complementary tools for improving the impact of non-invasive BCIs

Low-cost methodologies and devices applied to measure, model and self-regulate emotions for Human-Computer Interaction

En aquesta tesi s'exploren les diferents metodologies d'anàlisi de l'experiència UX des d'una visió centrada en usuari. Aquestes metodologies clàssiques i fonamentades només permeten extreure dades cognitives, és a dir les dades que l'usuari és capaç de comunicar de manera conscient. L'objectiu de la tesi és proposar un model basat en l'extracció de dades biomètriques per complementar amb dades emotives (i formals) la informació cognitiva abans esmentada. Aquesta tesi no és només teòrica, ja que juntament amb el model proposat (i la seva evolució) es mostren les diferents proves, validacions i investigacions en què s'han aplicat, sovint en conjunt amb grups de recerca d'altres àrees amb èxit.En esta tesis se exploran las diferentes metodologías de análisis de la experiencia UX desde una visión centrada en usuario. Estas metodologías clásicas y fundamentadas solamente permiten extraer datos cognitivos, es decir los datos que el usuario es capaz de comunicar de manera consciente. El objetivo de la tesis es proponer un modelo basado en la extracción de datos biométricos para complementar con datos emotivos (y formales) la información cognitiva antes mencionada. Esta tesis no es solamente teórica, ya que junto con el modelo propuesto (y su evolución) se muestran las diferentes pruebas, validaciones e investigaciones en la que se han aplicado, a menudo en conjunto con grupos de investigación de otras áreas con éxito.In this thesis, the different methodologies for analyzing the UX experience are explored from a user-centered perspective. These classical and well-founded methodologies only allow the extraction of cognitive data, that is, the data that the user is capable of consciously communicating. The objective of this thesis is to propose a methodology that uses the extraction of biometric data to complement the aforementioned cognitive information with emotional (and formal) data. This thesis is not only theoretical, since the proposed model (and its evolution) is complemented with the different tests, validations and investigations in which they have been applied, often in conjunction with research groups from other areas with success

Beyond the Bayley: Neurocognitive Assessments of Development During Infancy and Toddlerhood

The use of global, standardized instruments is conventional among clinicians and researchers interested in assessing neurocognitive development. Exclusively relying on these tests for evaluating effects may underestimate or miss specific effects on early cognition. The goal of this review is to identify alternative measures for possible inclusion in future clinical trials and interventions evaluating early neurocognitive development. The domains included for consideration are attention, memory, executive function, language and socio-emotional development. Although domain-based tests are limited, as psychometric properties have not yet been well-established, this review includes tasks and paradigms that have been reliably used across various developmental psychology laboratories

Modélisation des émotions de l’apprenant et interventions implicites pour les systèmes tutoriels intelligents

La modélisation de l’expérience de l’utilisateur dans les Interactions Homme-Machine est un enjeu important pour la conception et le développement des systèmes adaptatifs intelligents. Dans ce contexte, une attention particulière est portée sur les réactions émotionnelles de l’utilisateur, car elles ont une influence capitale sur ses aptitudes cognitives, comme la perception et la prise de décision. La modélisation des émotions est particulièrement pertinente pour les Systèmes Tutoriels Émotionnellement Intelligents (STEI). Ces systèmes cherchent à identifier les émotions de l’apprenant lors des sessions d’apprentissage, et à optimiser son expérience d’interaction en recourant à diverses stratégies d’interventions. Cette thèse vise à améliorer les méthodes de modélisation des émotions et les stratégies émotionnelles utilisées actuellement par les STEI pour agir sur les émotions de l’apprenant. Plus précisément, notre premier objectif a été de proposer une nouvelle méthode pour détecter l’état émotionnel de l’apprenant, en utilisant différentes sources d’informations qui permettent de mesurer les émotions de façon précise, tout en tenant compte des variables individuelles qui peuvent avoir un impact sur la manifestation des émotions. Pour ce faire, nous avons développé une approche multimodale combinant plusieurs mesures physiologiques (activité cérébrale, réactions galvaniques et rythme cardiaque) avec des variables individuelles, pour détecter une émotion très fréquemment observée lors des sessions d’apprentissage, à savoir l’incertitude. Dans un premier lieu, nous avons identifié les indicateurs physiologiques clés qui sont associés à cet état, ainsi que les caractéristiques individuelles qui contribuent à sa manifestation. Puis, nous avons développé des modèles prédictifs permettant de détecter automatiquement cet état à partir des différentes variables analysées, à travers l’entrainement d’algorithmes d’apprentissage machine. Notre deuxième objectif a été de proposer une approche unifiée pour reconnaître simultanément une combinaison de plusieurs émotions, et évaluer explicitement l’impact de ces émotions sur l’expérience d’interaction de l’apprenant. Pour cela, nous avons développé une plateforme hiérarchique, probabiliste et dynamique permettant de suivre les changements émotionnels de l'apprenant au fil du temps, et d’inférer automatiquement la tendance générale qui caractérise son expérience d’interaction à savoir : l’immersion, le blocage ou le décrochage. L’immersion correspond à une expérience optimale : un état dans lequel l'apprenant est complètement concentré et impliqué dans l’activité d’apprentissage. L’état de blocage correspond à une tendance d’interaction non optimale où l'apprenant a de la difficulté à se concentrer. Finalement, le décrochage correspond à un état extrêmement défavorable où l’apprenant n’est plus du tout impliqué dans l’activité d’apprentissage. La plateforme proposée intègre trois modalités de variables diagnostiques permettant d’évaluer l’expérience de l’apprenant à savoir : des variables physiologiques, des variables comportementales, et des mesures de performance, en combinaison avec des variables prédictives qui représentent le contexte courant de l’interaction et les caractéristiques personnelles de l'apprenant. Une étude a été réalisée pour valider notre approche à travers un protocole expérimental permettant de provoquer délibérément les trois tendances ciblées durant l’interaction des apprenants avec différents environnements d’apprentissage. Enfin, notre troisième objectif a été de proposer de nouvelles stratégies pour influencer positivement l’état émotionnel de l’apprenant, sans interrompre la dynamique de la session d’apprentissage. Nous avons à cette fin introduit le concept de stratégies émotionnelles implicites : une nouvelle approche pour agir subtilement sur les émotions de l’apprenant, dans le but d’améliorer son expérience d’apprentissage. Ces stratégies utilisent la perception subliminale, et plus précisément une technique connue sous le nom d’amorçage affectif. Cette technique permet de solliciter inconsciemment les émotions de l’apprenant, à travers la projection d’amorces comportant certaines connotations affectives. Nous avons mis en œuvre une stratégie émotionnelle implicite utilisant une forme particulière d’amorçage affectif à savoir : le conditionnement évaluatif, qui est destiné à améliorer de façon inconsciente l’estime de soi. Une étude expérimentale a été réalisée afin d’évaluer l’impact de cette stratégie sur les réactions émotionnelles et les performances des apprenants.Modeling the user’s experience within Human-Computer Interaction is an important challenge for the design and development of intelligent adaptive systems. In this context, a particular attention is given to the user’s emotional reactions, as they decisively influence his cognitive abilities, such as perception and decision-making. Emotion modeling is particularly relevant for Emotionally Intelligent Tutoring Systems (EITS). These systems seek to identify the learner’s emotions during tutoring sessions, and to optimize his interaction experience using a variety of intervention strategies. This thesis aims to improve current methods on emotion modeling, as well as the emotional strategies that are presently used within EITS to influence the learner’s emotions. More precisely, our first objective was to propose a new method to recognize the learner’s emotional state, using different sources of information that allow to measure emotions accurately, whilst taking account of individual characteristics that can have an impact on the manifestation of emotions. To that end, we have developed a multimodal approach combining several physiological measures (brain activity, galvanic responses and heart rate) with individual variables, to detect a specific emotion, which is frequently observed within computer tutoring, namely : uncertainty. First, we have identified the key physiological indicators that are associated to this state, and the individual characteristics that contribute to its manifestation. Then, we have developed predictive models to automatically detect this state from the analyzed variables, trough machine learning algorithm training. Our second objective was to propose a unified approach to simultaneously recognize a combination of several emotions, and to explicitly evaluate the impact of these emotions on the learner’s interaction experience. For this purpose, we have developed a hierarchical, probabilistic and dynamic framework, which allows one to track the learner’s emotional changes over time, and to automatically infer the trend that characterizes his interaction experience namely : flow, stuck or off-task. Flow is an optimal experience : a state in which the learner is completely focused and involved within the learning activity. The state of stuck is a non-optimal trend of the interaction where the learner has difficulty to maintain focused attention. Finally, the off-task behavior is an extremely unfavorable state where the learner is not involved anymore within the learning session. The proposed framework integrates three-modality diagnostic variables that sense the learner’s experience including : physiology, behavior and performance, in conjunction with predictive variables that represent the current context of the interaction and the learner’s personal characteristics. A human-subject study was conducted to validate our approach through an experimental protocol designed to deliberately elicit the three targeted trends during the learners’ interaction with different learning environments. Finally, our third objective was to propose new strategies to positively influence the learner’s emotional state, without interrupting the dynamics of the learning session. To this end, we have introduced the concept of implicit emotional strategies : a novel approach to subtly impact the learner’s emotions, in order to improve his learning experience. These strategies use the subliminal perception, and more precisely a technique known as affective priming. This technique aims to unconsciously solicit the learner’s emotions, through the projection of primes charged with specific affective connotations. We have implemented an implicit emotional strategy using a particular form of affective priming namely : the evaluative conditioning, which is designed to unconsciously enhance self-esteem. An experimental study was conducted in order to evaluate the impact of this strategy on the learners’ emotional reactions and performance

Effectiveness analysis of traditional and mixed reality simulations in medical training: a methodological approach for the assessment of stress, cognitive load and performance

La simulazione nell'educazione in medicina è considerata un metodo di formazione in grado di migliorare le competenze cliniche e il comportamento degli operatori sanitari e, di conseguenza, la qualità dell'assistenza per il paziente. Inoltre, l'utilizzo di nuove tecnologie come la Realtà Aumentata, offre ai discenti l'opportunità di esercitarsi in un ambiente immersivo. L'opportunità di sperimentare questo innovativo metodo didattico è efficace non solo nel ridurre il rischio di errori e approcci sbagliati ma anche nel provare ansia e stress simili a quelli avvertiti nella pratica reale. La sfida sta nel trovare il giusto equilibrio. I discenti devono infatti provare lo stesso stress che avvertirebbero lavorando ad un vero caso clinico ma, allo stesso tempo, devono essere controllati ed evitati possibili disturbi da stress post-traumatico, verificabili soprattutto nel campo della gestione delle emergenze (pronto soccorso). Inoltre, è fondamentale anche ottenere alte prestazioni e un apprendimento adeguato, evitando sovraccarichi cognitivi che influenzerebbero negativamente l’apprendimento. Tuttavia, ad oggi mancano ancora studi approfonditi sull'impatto che le simulazioni mediche hanno su stress, frustrazione, carico cognitivo e apprendimento dei discenti. Per questo motivo, l'obiettivo principale di questo studio è valutare l'efficacia del training tramite simulazione, analizzando prestazioni, ansia, stress e carico cognitivo durante simulazioni cliniche tradizionali (con manichino) ed avanzate (in realtà mista). A questo scopo, è stato sviluppato un approccio metodologico strutturato e completo per valutare le prestazioni, le condizioni emotive e cognitive degli studenti. Questo comprende l'acquisizione e l'analisi di parametri psicologici (valutazione soggettiva), segnali biometrici (valutazione oggettiva) e prestazioni. Questa indagine consente di evidenziare i punti deboli delle simulazioni e offre l'opportunità di definire utili linee guida per la riprogettazione e l'ottimizzazione delle stesse. La metodologia è stata applicata su tre casi studio: il primo si riferisce a simulazioni ad alta fedeltà per la gestione del paziente in pronto soccorso, il secondo si riferisce a simulazioni a bassa fedeltà per la pratica della rachicentesi. Per il terzo caso studio, è stato progettato e sviluppato un prototipo di simulatore in realtà mista per la rachicentesi, con l'obiettivo di migliorare il senso di realismo e immersione della simulazione a bassa fedeltà. 148 studenti sono stati coinvolti nei primi due casi studio osservazionali, mentre soltanto 36 studenti hanno preso parte allo studio pilota sulla simulazione in realtà mista. In tutti i casi di studio sono state effettuate analisi descrittive delle prestazioni, degli stati cognitivi ed emotivi. Per le simulazioni ad alta e bassa fedeltà, le analisi di regressione statistica hanno evidenziato quali variabili influenzano le prestazioni, lo stress e il carico cognitivo degli studenti. Per lo studio pilota sulla realtà mista, l'analisi della user experience ha sottolineato i limiti tecnici della nuova tecnologia.Simulation in medical education is considered a training method capable of improving clinical competence and practitioners’ behaviour, and, consequently quality of care and patient’s outcome. Moreover, the use of new technologies, such as augmented reality, offers to the learners the opportunity to engage themselves in an immersive environment. The opportunity to experiment with this innovative instructional method is effective not only in reducing the risk of errors and wrong approaches but also in experiencing anxiety and stress as in real practice. The challenge is to find the right stress balance: learners have to feel as if they were practicing in the real stressful clinical case, and, at the same time, post-traumatic stress disorders, verifiable especially in the emergency field, must be controlled and avoided. Moreover, it is fundamental also to obtain high performance and learning, thus avoiding cognitive overloads. However, extensive researches about the impact of medical simulations on students’ stress, frustration, cognitive load, and learning are still lacking. For this reason, the main objective of this study is to assess simulation training effectiveness by analysing performance, anxiety, stress, and cognitive load during traditional (with manikin) and advanced (with augmented reality) clinical simulations. A structured and comprehensive methodological approach to assess performance, emotional and cognitive conditions of students has been developed. It includes the acquisition and analysis of psychological parameters (subjective assessment), biometric signals (objective assessment), and task performance. This investigation allows to point out simulations’ weaknesses and offers the opportunity to define useful optimisation guidelines. The methodology has been applied to three case studies: the first one refers to high-fidelity simulations, for the patient management in the emergency room, the second one refers to low-fidelity simulation for rachicentesis. For the third case study, a prototype of a mixed reality simulator for the rachicentesis practice has been designed and developed aiming at improving the sense of realism and immersion of the low-fidelity simulation. While 148 students have been enrolled in the first two case studies, only 36 students have taken part in the pilot study about mixed reality simulation. Descriptive analysis about performance, cognitive and emotional states have been done in all the case studies. For the high-fidelity and low-fidelity simulations, the statistical regression analysis has pointed out which variables affect students’ performance, stress, and cognitive load. For the pilot study about mixed reality, the user experience analysis highlighted the technical limitations of the new technology