Towards unravelling the relationship between on-body, environmental and emotion data using sensor information fusion approach

Over the past few years, there has been a noticeable advancement in environmental models and information fusion systems taking advantage of the recent developments in sensor and mobile technologies. However, little attention has been paid so far to quantifying the relationship between environment changes and their impact on our bodies in real-life settings. In this paper, we identify a data driven approach based on direct and continuous sensor data to assess the impact of the surrounding environment and physiological changes and emotion. We aim at investigating the potential of fusing on-body physiological signals, environmental sensory data and on-line self-report emotion measures in order to achieve the following objectives: (1) model the short term impact of the ambient environment on human body, (2) predict emotions based on-body sensors and environmental data. To achieve this, we have conducted a real-world study ‘in the wild’ with on-body and mobile sensors. Data was collected from participants walking around Nottingham city centre, in order to develop analytical and predictive models. Multiple regression, after allowing for possible confounders, showed a noticeable correlation between noise exposure and heart rate. Similarly, UV and environmental noise have been shown to have a noticeable effect on changes in ElectroDermal Activity (EDA). Air pressure demonstrated the greatest contribution towards the detected changes in body temperature and motion. Also, significant correlation was found between air pressure and heart rate. Finally, decision fusion of the classification results from different modalities is performed. To the best of our knowledge this work presents the first attempt at fusing and modelling data from environmental and physiological sources collected from sensors in a real-world setting

Emotion recognition from physiological signals using fusion of wavelet based features

International audienceIn this paper we propose a new system for human emotion recognition based on multi resolution analysis of physiological signals. In our study we have used four kinds of bio signals EMG, RESP, ECG and SC recorded at the University of Augsburg. Daubechies Symlet, Haar and Morlet wavelet transform were applied to analyze the non-stationary signals. Physiological features was extracted from the most relevant wavelet coefficients and the feature vectors obtained from each signal were combined using multimodal fusion technique to construct one feature vector for each emotion. A support vector machine (SVM) was adopted as a pattern classifier, an improved recognition accuracy of 95% was obtained and it clearly proves the performance of our new wavelet based approach in emotion recognition

Multiresolution framework for emotion sensing in physiological signals

International audienceThis paper propose a new framework for emotion recognition and classification using a Continuous Wavelet Transform (CWT) for features extraction from physiological signal. Data from the emotional corpus recorded at Augsburg university were used in our study. In the first phase four wavelet families were chosen to analyze EMG RESP SC and ECG signals in order to extract emotional features in multi level of wavelet coefficients. The most relevant features vectors were combined to create a multimodal representation for each class of emotion. The proposed system was performed using an SVM classifier for the training and the test of the generated models, a classification accuracy of 95% was obtained and it clearly prove the performance of our framework in the estimation and characterization of emotional pattern