Detecting users’ cognitive load by galvanic skin response with affective interference

Experiencing high cognitive load during complex and demanding tasks results in performance reduction, stress, and errors. However, these could be prevented by a system capable of constantly monitoring users’ cognitive load fluctuations and adjusting its interactions accordingly. Physiological data and behaviors have been found to be suitable measures of cognitive load and are now available in many consumer devices. An advantage of these measures over subjective and performance-based methods is that they are captured in real time and implicitly while the user interacts with the system, which makes them suitable for real-world applications. On the other hand, emotion interference can change physiological responses and make accurate cognitive load measurement more challenging. In this work, we have studied six galvanic skin response (GSR) features in detection of four cognitive load levels with the interference of emotions. The data was derived from two arithmetic experiments and emotions were induced by displaying pleasant and unpleasant pictures in the background. Two types of classifiers were applied to detect cognitive load levels. Results from both studies indicate that the features explored can detect four and two cognitive load levels with high accuracy even under emotional changes. More specifically, rise duration and accumulative GSR are the common best features in all situations, having the highest accuracy especially in the presence of emotions

Using Noninvasive Brain Measurement to Explore the Psychological Effects of Computer Malfunctions on Users during Human-Computer Interactions

In today’s technologically driven world, there is a need to better understand the ways that common computer malfunctions affect computer users. These malfunctions may have measurable influences on computer user’s cognitive, emotional, and behavioral responses. An experiment was conducted where participants conducted a series of web search tasks while wearing functional nearinfrared spectroscopy (fNIRS) and galvanic skin response sensors. Two computer malfunctions were introduced during the sessions which had the potential to influence correlates of user trust and suspicion. Surveys were given after each session to measure user’s perceived emotional state, cognitive load, and perceived trust. Results suggest that fNIRS can be used to measure the different cognitive and emotional responses associated with computer malfunctions. These cognitive and emotional changes were correlated with users’ self-report levels of suspicion and trust, and they in turn suggest future work that further explores the capability of fNIRS for the measurement of user experience during human-computer interactions

The Effect of Emotional Distraction on Declarative Memory and an Exploration of its Physiological Marker: An Affective Computing Perspective

One of the advantage of e-learning method is the flexibility of embedding audio-visual materials, however whether this goal-irrelevant stimuli would distract the users instead and hinder their performance is generally unknown. The present study aimed to investigate the effects of emotional distraction (ED) on the declarative memory performance. The study was conducted using the within-subject experiment on 38 students aged 18-21 years (20 males). Declarative memory was measured using a Word-Pair Association (WPA) task. ED is given using the International Affective Pictures System (IAPS) and International Affective Digitized Sounds (IADS) grouped based on their valence (neutral, positive, and negative). Measurement of physiological responses was done by measuring Galvanic Skin Responses (GSR), and Electroencephalography (EEG) with Frontal Alpha Asymmetry (FAA) index. Result: significant difference on the WPA score of which Positive is lower than Neutral condition (p = 0.011), but only on the group which the positive block were presented first. From the GSR data, significant main effect of the order of experimental block regardless of the ED valence (p < 0.001; F = 16.045), the first block elicits significantly higher GSR amplitude compared to second (p < 0.001; t = 4.94) and third (p = 0.001; t = 3.90). Meanwhile, the FAA index showed no significant difference (p = 0.654; F = 0.433)

A wearable system for stress detection through physiological data analysis

In the last years the impact of stress on the society has been increased, resulting in 77% of people that regularly experiences physical symptoms caused by stress with a negative impact on their personal and professional life, especially in aging working population. This paper aims to demonstrate the feasibility of detection and monitoring of stress, inducted by mental stress tests, through the analysis of physiological data collected by wearable sensors. In fact, the physiological features extracted from heart rate variability and galvanic skin response showed significant differences between stressed and not stressed people. Starting from the physiological data, the work provides also a cluster analysis based on Principal Components (PCs) able to showed a visual discrimination of stressed and relaxed groups. The developed system would support active ageing, monitoring and managing the level of stress in ageing workers and allowing them to reduce the burden of stress related to the workload on the basis of personalized interventions

Stress level assessment with non-intrusive sensors

Mención Internacional en el título de doctorStress is an involuntary reaction where the human body changes from a calm state to an excited state in order to preserve the integrity of the organism. Small amount of stress should be good to became entrepreneur and learn new ways of thinking, but continuous stress can carry an array of daily risks, such as, cardiovascular diseases, hair loss, diabetes or immune dysregulation. Recognize how, when and where it occurs has become a step in stress assessment. Stress recognition starts from 1973 until now. This disease has become a problem in recent years because has increased the number of cases, especially in workers where his/her performance decreases. Stress reactions are provoked for the Autonomous Nervous System (ANS) and one way to estimate it could be found in physiological signals. A list of a variety wearable sensor is presented to capture these reactions, trying to minimize the risk of distraction due to external factors. The aim of this work thesis is to detect stress for level assessment. A combination of different physiological signals is selected to extract stress feature an classify in a rating scale from relax to breakdown situations. This thesis proposes a new feature extraction model to understand physiological Galvanic Skin Response (GSR) reactions. Last methods conclude in incongruent results that are not interpretable. This model propose a robust algorithm that can be used in real-time (low time computability) and results are sparse in time to obtain an easily statistical and graphical interpretation. Signal processing methods of heart rhythm and hormone cortisol are included to develop a robust feature extraction method of stress reactions. A combination of electrodermal, heart and hormone analysis is presented to know in real-time the state of the individual. These features have been selected because the acquisition is non-intrusive avoiding other factor such as distractions. This thesis is application-focused and highly multidisciplinary. A complete feature extraction model is presented including the new electrodermal model named and usual heart rhythm techniques. Three experiments were evaluated: a) a feature selection model using neurocognitive games, b) a stress classifier in time during public talks, and c) a real-time stress assessment classifier in a five-star rating scale. This thesis improve stress detection overcoming a system to capture physiological responses, analyze and conclude a stress assessment decision. We discussed past state of the art and propose a new method of feature extraction using signal processing improvements. Three different scenarios were evaluated to confirm the achievement of aims proposed.Programa Oficial de Doctorado en Multimedia y ComunicacionesPresidente: Joaquín Míguez Arenas.- Secretario: Luis Ignacio Santamaría Caballero.- Vocal: Mª Isabel Valera Martíne

Inside Out: Detecting Learners' Confusion to Improve Interactive Digital Learning Environments

Confusion is an emotion that is likely to occur while learning complex information. This emotion can be beneficial to learners in that it can foster engagement, leading to deeper understanding. However, if learners fail to resolve confusion, its effect can be detrimental to learning. Such detrimental learning experiences are particularly concerning within digital learning environments (DLEs), where a teacher is not physically present to monitor learner engagement and adapt the learning experience accordingly. However, with better information about a learner's emotion and behavior, it is possible to improve the design of interactive DLEs (IDLEs) not only in promoting productive confusion but also in preventing overwhelming confusion. This article reviews different methodological approaches for detecting confusion, such as self-report and behavioral and physiological measures, and discusses their implications within the theoretical framework of a zone of optimal confusion. The specificities of several methodologies and their potential application in IDLEs are discussed

Cognitive Heat: Exploring the Usage of Thermal Imaging to Unobtrusively Estimate Cognitive Load

Current digital systems are largely blind to users’ cognitive states. Systems that adapt to users’ states show great potential for augmenting cognition and for creating novel user experiences. However, most approaches for sensing cognitive states, and cognitive load specifically, involve obtrusive technologies, such as physiological sensors attached to users’ bodies. This paper present an unobtrusive indicator of the users’ cognitive load based on thermal imaging that is applicable in real-world. We use a commercial thermal camera to monitor a person’s forehead and nose temperature changes to estimate their cognitive load. To assess the effect of different levels of cognitive load on facial temperature we conducted a user study with 12 participants. The study showed that different levels of the Stroop test and the complexity of reading texts affect facial temperature patterns, thereby giving a measure of cognitive load. To validate the feasibility for real-time assessments of cognitive load, we conducted a second study with 24 participants, we analyzed the temporal latency of temperature changes. Our system detected temperature changes with an average latency of 0.7 seconds after users were exposed to a stimulus, outperforming latency in related work that used other thermal imaging techniques. We provide empirical evidence showing how to unobtrusively detect changes in cognitive load in real-time. Our exploration of exposing users to different content types gives rise to thermal-based activity tracking, which facilitates new applications in the field of cognition-aware computing

Monitoring Physiological Reactions of Construction Workers in Virtual Environment: A Feasibility Study Using Affective Sensing Technology

This research aims to monitor workers’ physiological reactions in virtual construction scenario. With the objective of leveraging affective sensing technology in construction scenario, experiments with Galvanic Skin Response (GSR) was conducted in a 3D simulation developed based on a real construction site. The GSR results obtained from sensor were analyzed in order (i) to assess the feasibility of using virtual environment to generate real emotions, (ii) to examine the relation between questionnaires used to ask people about their experience and their physiological responses and (iii) to identify the factors that affect people’s emotional reactions in virtual environment. Subjects of the experimental group exhibited incoherent responses, as expected in experiments with human subjects. Based on the various reasons for this incoherence obtained from questionnaire part of the experiment, the potential in research for developing training methods with respect to workers’ physiological response capability was identified