19,821 research outputs found

    Facial expression based emotion recognition using neural networks

    Get PDF
    Facial emotion recognition has been extensively studied over the last decade due to its various applications in the fields such as human-computer interaction and data analytics. In this paper, we develop a facial emotion recognition approach to classify seven emotional states (joy, sadness, surprise, anger, fear, disgust and neutral). Seventeen action units tracked by Kinect v2 sensor have been used as features. Classification of emotions was performed by artificial neural networks (ANNs). Six subjects took part in the experiment. We have achieved average accuracy of 95.8% for the case in which we tested our approach with the same volunteers took part in our data generation process. We also evaluated the performance of the network with additional volunteers who were not part of the training data and achieved 67.03% classification accuracy

    A Study of Three Artificial Neural Networks Models\u27 Ability to Identify Emotions from Facial Images

    Get PDF
    Facial expressions conveying emotions are vital for human communication. They are also important in the studies of human interaction and behavioral studies. Recognition of emotions, using facial images, may provide a fast and practical approach that is noninvasive. Most previous studies of emotion recognition through facial images were based on the Facial Action Coding System (FACS). The FACS, which was developed by Ekman and Freisen in 1978, was created to identify different facial muscular actions. Previous artificial neural network-based approaches for classification of facial expressions focused on improving one particular neural network model for better accuracy. The purpose of this present study was to compare different artificial neural network models, and determine which model was best at recognizing emotions through facial images. The three neural network models were: 1 . The Hopfield network 2. The Learning Vector Quantization network 3. Multilayer (Feedforward) back-propagation network Several facial parameters were extracted from facial images and used in training the different neural network models. The best performing neural network was the Hopfield network at 72.50% accuracy. Next, the facial parameters were tested for their significance in identifying facial expressions and a subset of the original facial parameters was used to retrain the networks. The best performing network using the subset of facial parameters was the LVQ network at 67.50% accuracy. This study has helped to understand which neural network model was best at identifying facial expression and to understand the importance of having a good set of parameters representing the facial expression. This study has shown that more research is needed to find a good set of parameters that will improve the accuracy of emotion identification using artificial neural networks

    Emotion Recognition System from Speech and Visual Information based on Convolutional Neural Networks

    Full text link
    Emotion recognition has become an important field of research in the human-computer interactions domain. The latest advancements in the field show that combining visual with audio information lead to better results if compared to the case of using a single source of information separately. From a visual point of view, a human emotion can be recognized by analyzing the facial expression of the person. More precisely, the human emotion can be described through a combination of several Facial Action Units. In this paper, we propose a system that is able to recognize emotions with a high accuracy rate and in real time, based on deep Convolutional Neural Networks. In order to increase the accuracy of the recognition system, we analyze also the speech data and fuse the information coming from both sources, i.e., visual and audio. Experimental results show the effectiveness of the proposed scheme for emotion recognition and the importance of combining visual with audio data

    Synch-Graph : multisensory emotion recognition through neural synchrony via graph convolutional networks

    Get PDF
    Human emotions are essentially multisensory, where emotional states are conveyed through multiple modalities such as facial expression, body language, and non-verbal and verbal signals. Therefore having multimodal or multisensory learning is crucial for recognising emotions and interpreting social signals. Existing multisensory emotion recognition approaches focus on extracting features on each modality, while ignoring the importance of constant interaction and co- learning between modalities. In this paper, we present a novel bio-inspired approach based on neural synchrony in audio- visual multisensory integration in the brain, named Synch-Graph. We model multisensory interaction using spiking neural networks (SNN) and explore the use of Graph Convolutional Networks (GCN) to represent and learn neural synchrony patterns. We hypothesise that modelling interactions between modalities will improve the accuracy of emotion recognition. We have evaluated Synch-Graph on two state- of-the-art datasets and achieved an overall accuracy of 98.3% and 96.82%, which are significantly higher than the existing techniques.Postprin

    Multi-Network Feature Fusion Facial Emotion Recognition using Nonparametric Method with Augmentation

    Get PDF
    Facial expression emotion identification and prediction is one of the most difficult problems in computer science. Pre-processing and feature extraction are crucial components of the more conventional methods. For the purpose of emotion identification and prediction using 2D facial expressions, this study targets the Face Expression Recognition dataset and shows the real implementation or assessment of learning algorithms such as various CNNs. Due to its vast potential in areas like artificial intelligence, emotion detection from facial expressions has become an essential requirement. Many efforts have been done on the subject since it is both a challenging and fascinating challenge in computer vision. The focus of this study is on using a convolutional neural network supplemented with data to build a facial emotion recognition system. This method may use face images to identify seven fundamental emotions, including anger, contempt, fear, happiness, neutrality, sadness, and surprise. As well as improving upon the validation accuracy of current models, a convolutional neural network that takes use of data augmentation, feature fusion, and the NCA feature selection approach may assist solve some of their drawbacks. Researchers in this area are focused on improving computer predictions by creating methods to read and codify facial expressions. With deep learning's striking success, many architectures within the framework are being used to further the method's efficacy. We highlight the contributions dealt with, the architecture and databases used, and demonstrate the development by contrasting the offered approaches and the outcomes produced. The purpose of this study is to aid and direct future researchers in the subject by reviewing relevant recent studies and offering suggestions on how to further the field. An innovative feature-based transfer learning technique is created using the pre-trained networks MobileNetV2 and DenseNet-201. The suggested system's recognition rate is 75.31%, which is a significant improvement over the results of the prior feature fusion study

    Robust Facial Emotion Recognition using Marine Predators Algorithm with Deep Convolutional Neural Network

    Get PDF
    Facial emotion recognition (FER) is a technology that includes the automatic identification and categorization of human sentiments depending on facial emotions. It leverages deep learning (DL), computer vision (CV), and machine learning (ML) methods to analyze the features of an individual's face, like the place of the mouth, eyes, eyebrows, and complete facial movements for determining their emotional conditions. Popular emotions that FER can recognize comprise surprise, sadness, happiness, fear, disgust, and anger. FER employing DL has an advanced application of artificial intelligence (AI) and deep neural networks (DNNs) that contain training methods to automatically recognize and categorize human expressions based on facial expressions. This approach normally comprises convolutional neural networks (CNNs) or highly complex models namely recurrent neural networks (RNNs) and convolutional RNNs (CRNNs) for analyzing and interpreting complex facial features and dynamics. This study introduces a new Robust Facial Emotion Recognition employing the Marine Predators Algorithm with Deep Learning (RFER-MPADL) approach. The main aim of the RFER-MPADL technique is to detect and categorize different kinds of emotions expressed in facial images. To accomplish this, the RFER-MPADL technique initially applies a bilateral filtering (BF) approach for the preprocessing step. Additionally, the RFER-MPADL technique uses the EfficientNet-B0 method for feature extraction. Moreover, the tuning process of the EfficientNet-B0 method was implemented using the MPA. Finally, the classification of facial emotions can be performed by the use of a deep autoencoder (DAE), in turn augments the overall performance of the RFER-MPADL method. The experimental analysis of the RFER-MPADL methodology is assessed on a standard facial expression dataset. The extensive outcomes exhibited the effectual performance of the RFER-MPADL methodology over other methods

    Emotion and Stress Recognition Related Sensors and Machine Learning Technologies

    Get PDF
    This book includes impactful chapters which present scientific concepts, frameworks, architectures and ideas on sensing technologies and machine learning techniques. These are relevant in tackling the following challenges: (i) the field readiness and use of intrusive sensor systems and devices for capturing biosignals, including EEG sensor systems, ECG sensor systems and electrodermal activity sensor systems; (ii) the quality assessment and management of sensor data; (iii) data preprocessing, noise filtering and calibration concepts for biosignals; (iv) the field readiness and use of nonintrusive sensor technologies, including visual sensors, acoustic sensors, vibration sensors and piezoelectric sensors; (v) emotion recognition using mobile phones and smartwatches; (vi) body area sensor networks for emotion and stress studies; (vii) the use of experimental datasets in emotion recognition, including dataset generation principles and concepts, quality insurance and emotion elicitation material and concepts; (viii) machine learning techniques for robust emotion recognition, including graphical models, neural network methods, deep learning methods, statistical learning and multivariate empirical mode decomposition; (ix) subject-independent emotion and stress recognition concepts and systems, including facial expression-based systems, speech-based systems, EEG-based systems, ECG-based systems, electrodermal activity-based systems, multimodal recognition systems and sensor fusion concepts and (x) emotion and stress estimation and forecasting from a nonlinear dynamical system perspective

    A novel driver emotion recognition system based on deep ensemble classification

    Get PDF
    Driver emotion classification is an important topic that can raise awareness of driving habits because many drivers are overconfident and unaware of their bad driving habits. Drivers will acquire insight into their poor driving behaviors and be better able to avoid future accidents if their behavior is automatically identified. In this paper, we use different models such as convolutional neural networks, recurrent neural networks, and multi-layer perceptron classification models to construct an ensemble convolutional neural network-based enhanced driver facial expression recognition model. First, the faces of the drivers are discovered using the faster region-based convolutional neural network (R-CNN) model, which can recognize faces in real-time and offline video reliably and effectively. The feature-fusing technique is utilized to integrate the features extracted from three CNN models, and the fused features are then used to train the suggested ensemble classification model. To increase the accuracy and efficiency of face detection, a new convolutional neural network block (InceptionV3) replaces the improved Faster R-CNN feature-learning block. To evaluate the proposed face detection and driver facial expression recognition (DFER) datasets, we achieved an accuracy of 98.01%, 99.53%, 99.27%, 96.81%, and 99.90% on the JAFFE, CK+, FER-2013, AffectNet, and custom-developed datasets, respectively. The custom-developed dataset has been recorded as the best among all under the simulation environment
    corecore