Multisensory Emotion Recognition With Speech and Facial Expression

Computers through both desktop and mobile devices are only becoming more important in our lives leading us to have more involved and longer interactions with them. Because of this our brains actually classify our involvement with them in a manner similar to our interactions with our fellow humans. This can lead to frustration and anxiety when our computers interrupt our work or pleasure with contextually inappropriate messages, much the same way it would if a friend or co-worker was pushy or rude. A way to solve this issue is to give our machines emotional intelligence, or the ability to recognize and be aware of our emotions. While monitoring physiological symptoms such as skin conductivity and muscle tension is one of the most accurate ways of detecting emotions, it can also be done in a more physically and socially comfortable manner by way of visual and auditory clues. This thesis will create a bimodal system where input is visual information via a still image and auditory information via a clip of human speech. The system will use two existing programs to identify the emotion found in each and, by using a weighted system, return the singular emotion felt

Understanding public speakers’ performance: first contributions to support a computational approach

Communication is part of our everyday life and our ability to communicate can have a significant role in a variety of contexts in our personal, academic, and professional lives. For long, the characterization of what is a good communicator has been subject to research and debate by several areas, particularly in Education, with a focus on improving the performance of teachers. In this context, the literature suggests that the ability to communicate is not only defined by the verbal component, but also by a plethora of non-verbal contributions providing redundant or complementary information, and, sometimes, being the message itself. However, even though we can recognize a good or bad communicator, objectively, little is known about what aspects – and to what extent—define the quality of a presentation. The goal of this work is to create the grounds to support the study of the defining characteristics of a good communicator in a more systematic and objective form. To this end, we conceptualize and provide a first prototype for a computational approach to characterize the different elements that are involved in communication, from audiovisual data, illustrating the outcomes and applicability of the proposed methods on a video database of public speakers.publishe

Emotion Recognition from Speech with Acoustic, Non-Linear and Wavelet-based Features Extracted in Different Acoustic Conditions

ABSTRACT: In the last years, there has a great progress in automatic speech recognition. The challenge now it is not only recognize the semantic content in the speech but also the called "paralinguistic" aspects of the speech, including the emotions, and the personality of the speaker. This research work aims in the development of a methodology for the automatic emotion recognition from speech signals in non-controlled noise conditions. For that purpose, different sets of acoustic, non-linear, and wavelet based features are used to characterize emotions in different databases created for such purpose