Analysis and detection of human emotion and stress from speech signals

Ph.DDOCTOR OF PHILOSOPH

Speech Recognition

Chapters in the first part of the book cover all the essential speech processing techniques for building robust, automatic speech recognition systems: the representation for speech signals and the methods for speech-features extraction, acoustic and language modeling, efficient algorithms for searching the hypothesis space, and multimodal approaches to speech recognition. The last part of the book is devoted to other speech processing applications that can use the information from automatic speech recognition for speaker identification and tracking, for prosody modeling in emotion-detection systems and in other speech processing applications that are able to operate in real-world environments, like mobile communication services and smart homes

A Survey on Human Emotion Recognition Approaches, Databases and Applications

This paper presents the various emotion classification and recognition systems which implement methods aiming at improving Human Machine Interaction. The modalities and approaches used for affect detection vary and contribute to accuracy and efficacy in detecting emotions of human beings. This paper discovers them in a comparison and descriptive manner. Various applications that use the methodologies in different contexts to address the challenges in real time are discussed. This survey also describes the databases that can be used as standard data sets in the process of emotion identification. Thus an integrated discussion of methods, databases used and applications pertaining to the emerging field of Affective Computing (AC) is done and surveyed.This paper presents the various emotion classification and recognition systems which implement methods aiming at improving Human Machine Interaction. The modalities and approaches used for affect detection vary and contribute to accuracy and efficacy in detecting emotions of human beings. This paper discovers them in a comparison and descriptive manner. Various applications that use the methodologies in different contexts to address the challenges in real time are discussed. This survey also describes the databases that can be used as standard data sets in the process of emotion identification. Thus an integrated discussion of methods, databases used and applications pertaining to the emerging field of Affective Computing (AC) is done and surveyed

Global affective computing research in the period 1997-2017: a bibliometric analysis

Notable fallouts in marketing and financial market prediction have raised the interest by the scientific community and the business world in Affective Computing (AfC). Automatically recognizing and responding to a user’s affective states, AfC shows a great potential to improve companies capabilities of customer relationship management. The aim of this study is to evaluate this field of research during the last twenty years, identifying for one side its evolution, by the major publications, citations, journals, authors, productive countries, productive institutions, and collaboration patterns; and for another side, identifying its trends through the analysis of research hotspots, burst keywords and areas of research done so far. This bibliometric analysis is based on the science citation index expanded (SCI-E), from the Institute of Scientific Information Web-of science, which is now firmly established as an integral part of research evaluation methodology especially within the scientific and applied fields. The results show a significant 4.19 rate of growth in AfC, doubling the number of publications in 4.02 years time. This field of interest is paving the way for creativity and innovation and provides opportunities for its greater development.info:eu-repo/semantics/acceptedVersio

Integrating Emotion Recognition Tools for Developing Emotionally Intelligent Agents

Emotionally responsive agents that can simulate emotional intelligence increase the acceptance of users towards them, as the feeling of empathy reduces negative perceptual feedback. This has fostered research on emotional intelligence during last decades, and nowadays numerous cloud and local tools for automatic emotional recognition are available, even for inexperienced users. These tools however usually focus on the recognition of discrete emotions sensed from one communication channel, even though multimodal approaches have been shown to have advantages over unimodal approaches. Therefore, the objective of this paper is to show our approach for multimodal emotion recognition using Kalman filters for the fusion of available discrete emotion recognition tools. The proposed system has been modularly developed based on an evolutionary approach so to be integrated in our digital ecosystems, and new emotional recognition sources can be easily integrated. Obtained results show improvements over unimodal tools when recognizing naturally displayed emotions

Predicting emotion in speech: a Deep Learning approach using Attention mechanisms

Speech Emotion Recognition (SER) has recently become a popular field of research because of its implications in human-computer interaction. In this study, the emotional state of the speaker is successfully predicted by using Deep Convolutional Neural Networks to automatically extract features from the spectrogram of a speech signal. Parting from a baseline model that uses a statistical approach to pooling, an alternative method is proposed by incorporating Attention mechanisms as a pooling strategy. Additionally, multi-task learning is explored as an improvement over the baseline model by assigning language recognition as an auxiliary task. The final results show a remarkable improvement in classification accuracy in respect to previous more conventional techniques, in particular Gaussian Mixture Models and i-vectors, as well as a notable improvement in performance of the proposed Attention mechanisms over statistical pooling.En las últimas décadas, Speech Emotion Recognition (SER), o el reconocimiento de emociones por voz, ha generado un fuerte interés en el ámbito del tratamiento del habla por sus implicaciones en la interacción humano-computador. En este trabajo, se consigue reconocer el estado emocional del hablante mediante redes convolucionales profundas, capaces de extraer de manera automática características contenidas en el espectrograma de la señal de voz. Partiendo de un modelo que utiliza análisis estadístico para pooling, se propone una estrategia alternativa para mejorar el rendimiento incorporando mecanismos de Atención. Como mejora añadida, se explora el campo del multi-task learning definiendo el reconocimiento del idioma como tasca auxiliar para el modelo. Los resultados obtenidos reflejan una mejora substancial en la precisión comparado con anteriores técnicas más convencionales, concretamente Gaussian Mixture Models y i-vectors, y una mejora notable en la precisión de los mecanismos de Atención respecto al pooling estadístico.En les últimes dècades, Speech Emotion Recognition (SER), o el Reconeixement d'Emocions per Veu, ha generat fort interès en l'àmbit del tractament de la parla per a les implicacions que presenta en la interacció humà-computador. En aquest treball s'aconsegueix reconèixer l'estat emocional del parlant utilitzant xarxes neuronals profundes que extreuen de manera automàtica característiques contingudes en l'espectrograma del senyal de veu. Partint d'un model que utilitza anàlisi estadística per a pooling, es proposa una estratègia alternativa per a millorar el rendiment incorporant mecanismes d'Atenció. Com a millora afegida, s'explora el camp del mulit-task learning definint el reconeixement de l'idioma com a tasca auxiliar per al model. Els resultats finals obtinguts reflecteixen una millora substancial en la precisió comparat amb anteriors mètodes, concretament respecte Gaussian Mixture Models i i-vectors, i una millora notable en la precisió dels mecanismes d'Atenció respecte el pooling estadístic