Multiple Instance Learning for Emotion Recognition using Physiological Signals

The problem of continuous emotion recognition has been the subject of several studies. The proposed affective computing approaches employ sequential machine learning algorithms for improving the classification stage, accounting for the time ambiguity of emotional responses. Modeling and predicting the affective state over time is not a trivial problem because continuous data labeling is costly and not always feasible. This is a crucial issue in real-life applications, where data labeling is sparse and possibly captures only the most important events rather than the typical continuous subtle affective changes that occur. In this work, we introduce a framework from the machine learning literature called Multiple Instance Learning, which is able to model time intervals by capturing the presence or absence of relevant states, without the need to label the affective responses continuously (as required by standard sequential learning approaches). This choice offers a viable and natural solution for learning in a weakly supervised setting, taking into account the ambiguity of affective responses. We demonstrate the reliability of the proposed approach in a gold-standard scenario and towards real-world usage by employing an existing dataset (DEAP) and a purposely built one (Consumer). We also outline the advantages of this method with respect to standard supervised machine learning algorithms

GSR Analysis for Stress: Development and Validation of an Open Source Tool for Noisy Naturalistic GSR Data

The stress detection problem is receiving great attention in related research communities. This is due to its essential part in behavioral studies for many serious health problems and physical illnesses. There are different methods and algorithms for stress detection using different physiological signals. Previous studies have already shown that Galvanic Skin Response (GSR), also known as Electrodermal Activity (EDA), is one of the leading indicators for stress. However, the GSR signal itself is not trivial to analyze. Different features are extracted from GSR signals to detect stress in people like the number of peaks, max peak amplitude, etc. In this paper, we are proposing an open-source tool for GSR analysis, which uses deep learning algorithms alongside statistical algorithms to extract GSR features for stress detection. Then we use different machine learning algorithms and Wearable Stress and Affect Detection (WESAD) dataset to evaluate our results. The results show that we are capable of detecting stress with the accuracy of 92 percent using 10-fold cross-validation and using the features extracted from our tool.Comment: 6 pages and 5 figures. Link to the github of the tool: https://github.com/HealthSciTech/pyED

Efficient emotion recognition using hyperdimensional computing with combinatorial channel encoding and cellular automata

In this paper, a hardware-optimized approach to emotion recognition based on the efficient brain-inspired hyperdimensional computing (HDC) paradigm is proposed. Emotion recognition provides valuable information for human-computer interactions, however the large number of input channels (>200) and modalities (>3) involved in emotion recognition are significantly expensive from a memory perspective. To address this, methods for memory reduction and optimization are proposed, including a novel approach that takes advantage of the combinatorial nature of the encoding process, and an elementary cellular automaton. HDC with early sensor fusion is implemented alongside the proposed techniques achieving two-class multi-modal classification accuracies of >76% for valence and >73% for arousal on the multi-modal AMIGOS and DEAP datasets, almost always better than state of the art. The required vector storage is seamlessly reduced by 98% and the frequency of vector requests by at least 1/5. The results demonstrate the potential of efficient hyperdimensional computing for low-power, multi-channeled emotion recognition tasks

Toolbox for Emotional feAture extraction from Physiological signals (TEAP)

Physiological response is an important component of an emotional episode. In this paper, we introduce a Toolbox for Emotional feAture Extraction from Physiological signals (TEAP). This open source toolbox can preprocess and calculate emotionally relevant features from multiple physiological signals, namely, electroencephalogram (EEG), galvanic skin response (GSR), electromyogram (EMG), skin temperature, respiration pattern, and blood volume pulse. The features from this toolbox are tested on two publicly available databases, i.e., MAHNOB-HCI and DEAP. We demonstrate that we achieve similar performance to the original work with the features from this toolbox. The toolbox is implemented in MATLAB and is also compatible with Octave. We hope this toolbox to be further developed and accelerate research in affective physiological signal analysis

Rehabilitative Movement Approaches and Dance Interventions in Parkinson’s Disease

The scope of this work is to address the functional deficits and symptoms experienced by those living with Parkinson’s Disease through movement interventions. Chapter 1 offers a brief overview of current pharmacotherapy and rehabilitation approaches in Parkinson’s, focusing on dance in particular as a movement intervention that may be particularly suited to this population. Chapter 2 focuses on brain plasticity and motor learning in PD, reporting the effects of rTMS applied after the acquisition of a motor skill. In this study, adaptation tested in patients with PD was comparable in the sham and TMS sessions, while retention indices tested on the following day were significantly lower in the sham compared to the TMS session in which retention indices were restored to the level of the controls. Chapter 3 explores biological markers that can be measured as proxies of brain plasticity, which may be affected by an intensive rehabilitation treatment including aerobic exercise, physical and occupational therapies. We tested whether exercise could improve motor function while also enhancing BDNF-TrkB signaling in lymphocytes. After MIRT, all patients showed improvement in motor function, TrkB interaction with NMDAR and BDNF-TrkB signaling. Chapter 4 explores mechanisms that may explain the efficacy of dance interventions on Parkinson’s symptoms. Internal and external cueing strategies, as well as affective and cognitive changes influence the manifestation of motor symptoms. Importantly, plasticity-enhancement though dance may be involved in the improvements of PD symptoms following dance practice. Chapter 5 presents a novel study in which the effects of dance in PD are compared to those of an exercise intervention of matched intensity, but lacking dance elements such as music, metaphorical language, and social reality of grace and beauty. This study shows changes in physiology, affect, self-efficacy and motor performance. Chapter 6 summarizes all these findings while tracing a link between studies on neuroplasticity, exercise-based rehabilitation in PD, and the role of dance in affecting motor performance through mechanism of motivation and modulation of attention

Development of cognitive workload models to detect driving impairment

Tesi redactada en castellàDriving a vehicle is a complex activity exposed to continuous changes such as speed limits and vehicular traffic. Drivers require a high degree of concentration when performing this activity, increasing the amount of mental demand known as cognitive workload, causing vehicular accidents to the minimum negligence. In fact, human error is the leading contributing factor in over 90% of road accidents. In recent years, the subjects' cognitive workload levels while driving a vehicle have been predicted using subjective and vehicle performance tools. Other research has emphasized the use and analysis of physiological information, where electroencephalographic (EEG) signals are the most used to identify cognitive states due to their high precision. Although significant progress has been made in this area, these investigations have been based on traditional techniques or data analysis from a specific source due to the information's complexity. A new trend has been opened in the study of the internal behavior of subjects by implementing machine learning techniques to analyze information from various sources. However, there are still several challenges to face in this new line of research. This doctoral thesis presents a new model to predict the states of low and high cognitive workload of subjects when facing scenarios of driving a vehicle called GALoRSI-SVMRBF (Genetic Algorithms and Logistic Regression for the Structuring of Information-Support Vector Machine with Radial Basis Function Kernel). GALoRSI-SVMRBF is developed using machine learning algorithms based on information from EEG signals. Also, the information collected from NASA-TLX, instant online self-assessment and the error rate measure are implemented in the model. First, GALoRSI-SVMRBF proposes a new method for pattern recognition based on feature selection that combines statistical tests, genetic algorithms, and logistic regression. This method consists mainly of selecting an EEG dataset and exploring the information to identify the key features that recognize cognitive states. The selected data are defined as an index for pattern recognition and used to structure a new dataset capable of optimizing the model's learning and classification process. Second, the methodology and development of a classifier for the prediction model are presented, implementing machine learning algorithms. The classifier is developed mainly in two phases, defined as training and testing. Once the prediction model has been developed, this thesis presents the validation phase of GALoRSI-SVMRBF. The validation consists of evaluating the model's adaptability to new datasets, maintaining a high prediction rate. Finally, an analysis of the performance of GALoRSI-SVMRBF is presented. The objective is to know the model's scope and limitations, evaluating various performance metrics to find the optimal configuration for GALoRSI-SVMRBF. We found that GALoRSI-SVMRBF successfully predicts low and high cognitive workload of subjects while driving a vehicle. In general, it is observed that the model uses the information extracted from multiple EEG signals, reducing the original dataset by more than 50%, maximizing its predictive capacity, achieving a precision rate of >90% in the classification of the information. During this thesis, the experiments showed that obtaining a high percentage of prediction depends on several factors, from applying a useful collection technique data until the last step of the prediction model.La conducción de un vehículo es una actividad compleja que está expuesta a demandas que cambian continuamente por diferentes factores, tales como, el límite de velocidad, obstáculos en la vía, tráfico vehicular, entre otros. Al desempeñar esta actividad, los conductores requieren un alto grado de concentración incrementando la cantidad de demanda mental conocida como carga. En los últimos años, se han propuesto mecanismos para monitorear y/o predecir los niveles de carga cognitiva de los sujetos al conducir un vehículo, centrándose en el uso de herramientas subjetivas y de rendimiento vehicular. Otras investigaciones, han enfatizado en el uso y análisis de la información fisiológica, siendo las señales electroencefalográficas (EEG) las más utilizadas para identificar los estados cognitivos por su alta precisión. A pesar del gran avance realizado, estas investigaciones se han basado en técnicas tradicionales o en el análisis de la información proveniente de fuentes específicas para identificar el estado interno del sujeto, obteniendo modelos sobreentrenados o robustos, incrementando el tiempo de análisis afectando el desempeño del modelo. En esta tesis doctoral se presenta un nuevo modelo para predecir los estados de baja y alta carga cognitiva de los sujetos al enfrentarse a escenarios de la conducción de un vehículo denominado GALoRSI-SVMRBF (Genetic Algorithms and Logistic Regression for the Structuring of Information-Support Vector Machine with Radial Basis Function Kernel). GALoRSI-SVMRBF fue desarrollado utilizando los algoritmos de aprendizaje automático y técnicas estadísticas basado en la información proveniente de las señales EEG. Primero, GALoRSI-SVMRBF crea una base de datos extrayendo las características que serán utilizadas en el modelo a través de técnicas estadísticas. Posteriormente, propone un nuevo método para el reconocimiento de patrones basado en la selección de características que combina pruebas estadísticas, algoritmos genéticos y regresión logística. Este método consiste principalmente en seleccionar un conjunto de datos EEG y explorar la combinación de la información para identificar las características claves que contribuyan al reconocimiento de dos estados cognitivos. Después, la información seleccionada es definida como un índice para el reconocimiento de patrones y utilizada para estructurar un nuevo conjunto de datos que soporta información de uno o múltiples canales para optimizar el proceso de aprendizaje y clasificación del modelo. Por último, es desarrollado el clasificador del modelo de predicciones el cual consiste en dos etapas definidas como entrenamiento y prueba. Nosotros encontramos que GALoRSI-SVMRBF predice de manera exitosa la carga cognitiva baja y alta de los sujetos durante la conducción de un vehículo. En general, se observó que el modelo utiliza la información extraída de una o múltiples señales EEG y logrando una tasa de precisión >90% en la clasificación de la informaciónPostprint (published version