Affective Man-Machine Interface: Unveiling human emotions through biosignals

As is known for centuries, humans exhibit an electrical profile. This profile is altered through various psychological and physiological processes, which can be measured through biosignals; e.g., electromyography (EMG) and electrodermal activity (EDA). These biosignals can reveal our emotions and, as such, can serve as an advanced man-machine interface (MMI) for empathic consumer products. However, such a MMI requires the correct classification of biosignals to emotion classes. This chapter starts with an introduction on biosignals for emotion detection. Next, a state-of-the-art review is presented on automatic emotion classification. Moreover, guidelines are presented for affective MMI. Subsequently, a research is presented that explores the use of EDA and three facial EMG signals to determine neutral, positive, negative, and mixed emotions, using recordings of 21 people. A range of techniques is tested, which resulted in a generic framework for automated emotion classification with up to 61.31% correct classification of the four emotion classes, without the need of personal profiles. Among various other directives for future research, the results emphasize the need for parallel processing of multiple biosignals

Feature Space Augmentation: Improving Prediction Accuracy of Classical Problems in Cognitive Science and Computer Vison

The prediction accuracy in many classical problems across multiple domains has seen a rise since computational tools such as multi-layer neural nets and complex machine learning algorithms have become widely accessible to the research community. In this research, we take a step back and examine the feature space in two problems from very different domains. We show that novel augmentation to the feature space yields higher performance. Emotion Recognition in Adults from a Control Group: The objective is to quantify the emotional state of an individual at any time using data collected by wearable sensors. We define emotional state as a mixture of amusement, anger, disgust, fear, sadness, anxiety and neutral and their respective levels at any time. The generated model predicts an individual’s dominant state and generates an emotional spectrum, 1x7 vector indicating levels of each emotional state and anxiety. We present an iterative learning framework that alters the feature space uniquely to an individual’s emotion perception, and predicts the emotional state using the individual specific feature space. Hybrid Feature Space for Image Classification: The objective is to improve the accuracy of existing image recognition by leveraging text features from the images. As humans, we perceive objects using colors, dimensions, geometry and any textual information we can gather. Current image recognition algorithms rely exclusively on the first 3 and do not use the textual information. This study develops and tests an approach that trains a classifier on a hybrid text based feature space that has comparable accuracy to the state of the art CNN’s while being significantly inexpensive computationally. Moreover, when combined with CNN’S the approach yields a statistically significant boost in accuracy. Both models are validated using cross validation and holdout validation, and are evaluated against the state of the art

A Framework for Students Profile Detection

Some of the biggest problems tackling Higher Education Institutions are students’ drop-out and academic disengagement. Physical or psychological disabilities, social-economic or academic marginalization, and emotional and affective problems, are some of the factors that can lead to it. This problematic is worsened by the shortage of educational resources, that can bridge the communication gap between the faculty staff and the affective needs of these students. This dissertation focus in the development of a framework, capable of collecting analytic data, from an array of emotions, affects and behaviours, acquired either by human observations, like a teacher in a classroom or a psychologist, or by electronic sensors and automatic analysis software, such as eye tracking devices, emotion detection through facial expression recognition software, automatic gait and posture detection, and others. The framework establishes the guidance to compile the gathered data in an ontology, to enable the extraction of patterns outliers via machine learning, which assist the profiling of students in critical situations, like disengagement, attention deficit, drop-out, and other sociological issues. Consequently, it is possible to set real-time alerts when these profiles conditions are detected, so that appropriate experts could verify the situation and employ effective procedures. The goal is that, by providing insightful real-time cognitive data and facilitating the profiling of the students’ problems, a faster personalized response to help the student is enabled, allowing academic performance improvements

Beyond mobile apps: a survey of technologies for mental well-being

Mental health problems are on the rise globally and strain national health systems worldwide. Mental disorders are closely associated with fear of stigma, structural barriers such as financial burden, and lack of available services and resources which often prohibit the delivery of frequent clinical advice and monitoring. Technologies for mental well-being exhibit a range of attractive properties, which facilitate the delivery of state-of-the-art clinical monitoring. This review article provides an overview of traditional techniques followed by their technological alternatives, sensing devices, behaviour changing tools, and feedback interfaces. The challenges presented by these technologies are then discussed with data collection, privacy, and battery life being some of the key issues which need to be carefully considered for the successful deployment of mental health toolkits. Finally, the opportunities this growing research area presents are discussed including the use of portable tangible interfaces combining sensing and feedback technologies. Capitalising on the data these ubiquitous devices can record, state of the art machine learning algorithms can lead to the development of robust clinical decision support tools towards diagnosis and improvement of mental well-being delivery in real-time

A survey of comics research in computer science

Graphical novels such as comics and mangas are well known all over the world. The digital transition started to change the way people are reading comics, more and more on smartphones and tablets and less and less on paper. In the recent years, a wide variety of research about comics has been proposed and might change the way comics are created, distributed and read in future years. Early work focuses on low level document image analysis: indeed comic books are complex, they contains text, drawings, balloon, panels, onomatopoeia, etc. Different fields of computer science covered research about user interaction and content generation such as multimedia, artificial intelligence, human-computer interaction, etc. with different sets of values. We propose in this paper to review the previous research about comics in computer science, to state what have been done and to give some insights about the main outlooks

Intrinsic monitoring of learning success facilitates memory encoding via the activation of the SN/VTA-Hippocampal loop

Humans constantly learn in the absence of explicit rewards. However, the neurobiological mechanisms supporting this type of internally-guided learning (without explicit feedback) are still unclear. Here, participants who completed a task in which no external reward/feedback was provided, exhibited enhanced fMRI-signals within the dopaminergic midbrain, hippocampus, and ventral striatum (the SN/VTA-Hippocampal loop) when successfully grasping the meaning of new-words. Importantly, new-words that were better remembered showed increased activation and enhanced functional connectivity between the midbrain, hippocampus, and ventral striatum. Moreover, enhanced emotion-related physiological measures and subjective pleasantness ratings during encoding were associated with remembered new-words after 24 hr. Furthermore, increased subjective pleasantness ratings were also related to new-words remembered after seven days. These results suggest that intrinsic-potentially reward-related-signals, triggered by self-monitoring of correct performance, can promote the storage of new information into long-term memory through the activation of the SN/VTA-Hippocampal loop, possibly via dopaminergic modulation of the midbrain

Stress level assessment with non-intrusive sensors

Mención Internacional en el título de doctorStress is an involuntary reaction where the human body changes from a calm state to an excited state in order to preserve the integrity of the organism. Small amount of stress should be good to became entrepreneur and learn new ways of thinking, but continuous stress can carry an array of daily risks, such as, cardiovascular diseases, hair loss, diabetes or immune dysregulation. Recognize how, when and where it occurs has become a step in stress assessment. Stress recognition starts from 1973 until now. This disease has become a problem in recent years because has increased the number of cases, especially in workers where his/her performance decreases. Stress reactions are provoked for the Autonomous Nervous System (ANS) and one way to estimate it could be found in physiological signals. A list of a variety wearable sensor is presented to capture these reactions, trying to minimize the risk of distraction due to external factors. The aim of this work thesis is to detect stress for level assessment. A combination of different physiological signals is selected to extract stress feature an classify in a rating scale from relax to breakdown situations. This thesis proposes a new feature extraction model to understand physiological Galvanic Skin Response (GSR) reactions. Last methods conclude in incongruent results that are not interpretable. This model propose a robust algorithm that can be used in real-time (low time computability) and results are sparse in time to obtain an easily statistical and graphical interpretation. Signal processing methods of heart rhythm and hormone cortisol are included to develop a robust feature extraction method of stress reactions. A combination of electrodermal, heart and hormone analysis is presented to know in real-time the state of the individual. These features have been selected because the acquisition is non-intrusive avoiding other factor such as distractions. This thesis is application-focused and highly multidisciplinary. A complete feature extraction model is presented including the new electrodermal model named and usual heart rhythm techniques. Three experiments were evaluated: a) a feature selection model using neurocognitive games, b) a stress classifier in time during public talks, and c) a real-time stress assessment classifier in a five-star rating scale. This thesis improve stress detection overcoming a system to capture physiological responses, analyze and conclude a stress assessment decision. We discussed past state of the art and propose a new method of feature extraction using signal processing improvements. Three different scenarios were evaluated to confirm the achievement of aims proposed.Programa Oficial de Doctorado en Multimedia y ComunicacionesPresidente: Joaquín Míguez Arenas.- Secretario: Luis Ignacio Santamaría Caballero.- Vocal: Mª Isabel Valera Martíne