Inside Out: Detecting Learners' Confusion to Improve Interactive Digital Learning Environments

Confusion is an emotion that is likely to occur while learning complex information. This emotion can be beneficial to learners in that it can foster engagement, leading to deeper understanding. However, if learners fail to resolve confusion, its effect can be detrimental to learning. Such detrimental learning experiences are particularly concerning within digital learning environments (DLEs), where a teacher is not physically present to monitor learner engagement and adapt the learning experience accordingly. However, with better information about a learner's emotion and behavior, it is possible to improve the design of interactive DLEs (IDLEs) not only in promoting productive confusion but also in preventing overwhelming confusion. This article reviews different methodological approaches for detecting confusion, such as self-report and behavioral and physiological measures, and discusses their implications within the theoretical framework of a zone of optimal confusion. The specificities of several methodologies and their potential application in IDLEs are discussed

Psychophysiology-based QoE assessment : a survey

We present a survey of psychophysiology-based assessment for quality of experience (QoE) in advanced multimedia technologies. We provide a classification of methods relevant to QoE and describe related psychological processes, experimental design considerations, and signal analysis techniques. We summarize multimodal techniques and discuss several important aspects of psychophysiology-based QoE assessment, including the synergies with psychophysical assessment and the need for standardized experimental design. This survey is not considered to be exhaustive but serves as a guideline for those interested to further explore this emerging field of research

Vicarious Fear Learning: The Role of Empathy

Fear learning can take place indirectly, by observing others, as well as directly through personal experience. This study aimed to determine whether we could detect indirect – sometimes called vicarious – fear learning in the laboratory, and to examine the influence of trait empathy on the robustness of this learning experience. Deficient empathy features prominently in certain externalizing psychopathologies, and fear learning is theoretically implicated in many psychological disorders, but little research has examined the possible connection between the two. In the present study, we first showed participants (N = 80; Mage = 19.1 years, SD = 2.1; 62.5% white) a video of a stranger (called the demonstrator) undergoing a Pavlovian fear conditioning procedure, receiving shocks and loud noises in the context of one of two conditioned stimuli. Next, we presented participants with those same conditioned stimuli. Correlation analyses showed an association between participants’ autonomic nervous system reactivity while observing the demonstrator receive aversive stimuli and their own reactivity when presented with the same conditioned stimulus associated with shock in the video. Associations between trait empathy (measured with the Interpersonal Reactivity Index and the Affective and Cognitive Measure of Empathy) and the strength of vicarious fear learning were generally small. Our results suggest the independence of the emotional response and trait empathy during the observation of someone’s distress

Psychophysiological and behavioral responses to descriptive labels in modern art museums

Educational tools in art exhibitions seem crucial to improve the cultural and aesthetic experience, particularly of non-expert visitors, thus becoming a strategic goal for museums. However, there has not been much research regarding the impact of labels on the quality of visitors’ aesthetic experience. Therefore, here we compared the impact on the cognitive and emotional experience of naïve visitors between essential and descriptive labels, through multiple objective and subjective measurements, focusing on the controversial modern art museum context. We found that, after detailed descriptions, observers spend more time inspecting artworks, their eyes wander more looking for the described elements, their skin conductance and pupil size increase, and overall, they find the content less complex and more arousing. Our findings show that people do receive important benefits from reading detailed information about artworks. This suggests that elaborating effective labels should be a primary goal for museums interested in attracting a non-expert public

Low-cost methodologies and devices applied to measure, model and self-regulate emotions for Human-Computer Interaction

En aquesta tesi s'exploren les diferents metodologies d'anàlisi de l'experiència UX des d'una visió centrada en usuari. Aquestes metodologies clàssiques i fonamentades només permeten extreure dades cognitives, és a dir les dades que l'usuari és capaç de comunicar de manera conscient. L'objectiu de la tesi és proposar un model basat en l'extracció de dades biomètriques per complementar amb dades emotives (i formals) la informació cognitiva abans esmentada. Aquesta tesi no és només teòrica, ja que juntament amb el model proposat (i la seva evolució) es mostren les diferents proves, validacions i investigacions en què s'han aplicat, sovint en conjunt amb grups de recerca d'altres àrees amb èxit.En esta tesis se exploran las diferentes metodologías de análisis de la experiencia UX desde una visión centrada en usuario. Estas metodologías clásicas y fundamentadas solamente permiten extraer datos cognitivos, es decir los datos que el usuario es capaz de comunicar de manera consciente. El objetivo de la tesis es proponer un modelo basado en la extracción de datos biométricos para complementar con datos emotivos (y formales) la información cognitiva antes mencionada. Esta tesis no es solamente teórica, ya que junto con el modelo propuesto (y su evolución) se muestran las diferentes pruebas, validaciones e investigaciones en la que se han aplicado, a menudo en conjunto con grupos de investigación de otras áreas con éxito.In this thesis, the different methodologies for analyzing the UX experience are explored from a user-centered perspective. These classical and well-founded methodologies only allow the extraction of cognitive data, that is, the data that the user is capable of consciously communicating. The objective of this thesis is to propose a methodology that uses the extraction of biometric data to complement the aforementioned cognitive information with emotional (and formal) data. This thesis is not only theoretical, since the proposed model (and its evolution) is complemented with the different tests, validations and investigations in which they have been applied, often in conjunction with research groups from other areas with success

THE RELATIONSHIPS AMONG MASTER’S LEVEL COUNSELING TRAINEES’ TRAINING LEVEL, EMOTIONAL INTELLIGENCE, AND PSYCHOPHYSIOLOGICAL CORRELATES OF EMOTION REGULATION DURING A SIMULATED COUNSELING INTERACTION

This study explored the relationships among master’s level counseling trainees’ level of training, ability emotional intelligence (EI), and psychophysiological correlates of emotion regulation recorded during a video-simulated client interaction. Agreement exists among counselor educators, researchers, and theorists that counselors’ emotion regulation is foundational to the competent delivery of counseling treatment. The literature further suggests that counselors and trainees experience frequent emotional challenges that overwhelm emotion regulation skills, interfere with competent delivery of service, and affect client outcomes. However, little research in counseling training and supervision has investigated trainees’ emotion regulation or factors that support adaptive emotion regulation while trainees interact with clients who are experiencing emotional distress. Participants were 66 master’s level counseling trainees from counseling programs accredited by the Counsel for Accreditation of Counseling and Related Educational Programs. Participants’ EI was operationalized as scores on the Mayer, Salovey, and Caruso Emotional Intelligence Test (Mayer, Salovey, Caruso, & Sitarenios, 2003). Emotion regulation was operationalized as electrodermal activity (EDA), high-frequency heart rate variability, and the standard deviation of normal heartbeat intervals (HRV-SDNN). Correlation and regression analyses indicated that psychophysiological correlates of trainees’ emotion regulation were not significantly correlated with training. However, HRV-SDNN significantly correlated with total EI, and the EI subscale Perceiving Emotions, while EDA significantly correlated with the Managing Emotions subscale

Finding the Hidden: Detecting Atypical Affective States from Physiological Signals

In cognitive science, intuition is described as a strategy of processing information that relies on people's instinctive and emotional criteria. When compared with the deliberate choices made after conscious reasoning, the quick and intuitive decision making strategies can be more effective. The intuitive thinking provokes changes in human physiological responses which can be measured by sensors. Utilising physiological reactions, previous work shows that atypical patterns such as emotion expressions and image manipulations can be identified. This thesis expands the exploration to examine whether more atypical human behaviour can be recognised from physiological signals. The examined subtly atypical behaviour includes depression, doubt and deception, Depression is a serious chronic mental disease and is considered as an atypical health condition in people. Doubt is defined as a non-deliberate attempt to mislead others and is a passive form of deception, representing an atypicality from honest behaviours. Deception is a more purposeful attempt to deceive, and thus is a distinct type of atypicality than honest communication. Through examining physiological reactions from presenters who have a particular atypical behaviour or condition, and observers who view behaviours of presenters, this research aims to recognise atypicality in human behaviour. A collection of six user studies are conducted. In two user studies, presenters are asked to conduct doubting and deceiving behaviours, while the remaining user studies involve observers watching behaviours of presenters who suffer from depression, have doubt, or have conducted deception. Physiological reactions of both presenters and observers are collected, including Blood Volume Pulse, Electrodermal Activity, Skin Temperature and Pupillary Responses. Observers are also asked to explicitly evaluate whether the viewed presenters were being depressed, doubting, or deceiving. Investigations upon physiological data in this thesis finds that detectable cues corresponding with depression, doubt and deception can be found. Viewing depression provokes visceral physiological reactions in observers that can be measured. Such physiological responses can be used to derive features for machine learning models to accurately distinguish between healthy individuals and people with depression. By contrast, depression does not provoke strong conscious recognition in observers, resulting in a conscious evaluation accuracy slightly above chance level. Similar results are also found in detecting doubt and deception. People with doubt and deceit elicit consistent physiological reactions within themselves. These bodily responses can be utilised by machine learning models or deep learning models to recognise doubt or deception. The doubt and deceit in presenters can also be recognised using physiological signals in observers, with excellent recognition rates which are higher when compared with the conscious judgments from the same group of observers. The results indicate that atypicality in presenters can both be captured by physiological signals of presenters and observers. Presenters' physiological reactions contribute to higher recognition of atypicality, but observers' physiological responses can serve as a comparable alternative. The awareness of atypicality among observers happens physiologically, so can be used by machine learning models, even when they do not reach the consciousness of the person. The research findings lead to a further discussion around the implications of observers' physiological responses. Decision support applications which utilise a quantifiable measure of people's unconscious and intuitive 'gut feeling' can be developed based on the work reported here to assist people with medical diagnosis, information credibility evaluation, and criminal detection. Further research suggests exploring more atypical behaviours in the wild

iFocus: A Framework for Non-intrusive Assessment of Student Attention Level in Classrooms

The process of learning is not merely determined by what the instructor teaches, but also by how the student receives that information. An attentive student will naturally be more open to obtaining knowledge than a bored or frustrated student. In recent years, tools such as skin temperature measurements and body posture calculations have been developed for the purpose of determining a student\u27s affect, or emotional state of mind. However, measuring eye-gaze data is particularly noteworthy in that it can collect measurements non-intrusively, while also being relatively simple to set up and use. This paper details how data obtained from such an eye-tracker can be used to predict a student\u27s attention as a measure of affect over the course of a class. From this research, an accuracy of 77% was achieved using the Extreme Gradient Boosting technique of machine learning. The outcome indicates that eye-gaze can be indeed used as a basis for constructing a predictive model