Spatio-Temporal Facial Expression Recognition Using Convolutional Neural Networks and Conditional Random Fields

Automated Facial Expression Recognition (FER) has been a challenging task for decades. Many of the existing works use hand-crafted features such as LBP, HOG, LPQ, and Histogram of Optical Flow (HOF) combined with classifiers such as Support Vector Machines for expression recognition. These methods often require rigorous hyperparameter tuning to achieve good results. Recently Deep Neural Networks (DNN) have shown to outperform traditional methods in visual object recognition. In this paper, we propose a two-part network consisting of a DNN-based architecture followed by a Conditional Random Field (CRF) module for facial expression recognition in videos. The first part captures the spatial relation within facial images using convolutional layers followed by three Inception-ResNet modules and two fully-connected layers. To capture the temporal relation between the image frames, we use linear chain CRF in the second part of our network. We evaluate our proposed network on three publicly available databases, viz. CK+, MMI, and FERA. Experiments are performed in subject-independent and cross-database manners. Our experimental results show that cascading the deep network architecture with the CRF module considerably increases the recognition of facial expressions in videos and in particular it outperforms the state-of-the-art methods in the cross-database experiments and yields comparable results in the subject-independent experiments.Comment: To appear in 12th IEEE Conference on Automatic Face and Gesture Recognition Worksho

Implementasi Convolutional Neural Networks (CNN) untuk Klasifikasi Ekspresi Citra Wajah pada FER-2013 Dataset

Abstract - session recognition is an interesting topic, where facial expressions in today's technological advances can support several fields such as health, business, and so on. Facial expression recognition can be done using the extraction of certain features. Meanwhile, Convolutional Neural Network (CNN) can recognize an object in the image through the features found by itself in the convolution process. By using CNN's advantages, this study aims to see CNN's performance in facial expressions of happiness and sadness in unideal data conditions. So based on this research, on the FER2013 dataset, CNN using the Adamax optimizer produced a fairly good performance where the value is given is 66% compared to Adam, N-Adam, and SGD.Keywords  -   CNN, Facial Expression, FER-2013 Abstrak – Pengenalan ekspresi merupakan topik penelitian yang menarik, dimana peran ekspresi wajah dalam kemajuan teknologi saat ini dapat mendukung beberapa bidang seperti kesehatan, bisnis, dan sebagainya. Pengenalan wajah dapat dilakukan dengan menggunakan ekstraksi fitur-fitur tertentu. Sementara itu, Convolutional Neural Network (CNN) dapat mengenali objek pada citra melalui fitur yang ditemukannya sendiri dalam proses konvolusinya. Dengan menggunakan keunggulan CNN, maka penelitian ini bertujuan untuk mengetahui performa CNN dalam mengenali ekspresi wajah bahagia (happy) dan sedih (sad) pada kondisi data tidak ideal. Maka berdasarkan hasil penelitian ini, pada dataset FER2013, CNN dengan menggunakan Adamax optimizer menghasilkan performa yang cukup baik dimana akurasi yang diberikan adalah sebesar 66% dibandingkan dengan Adam, N-Adam, dan SGD.Kata Kunci - CNN, Ekspresi Wajah, FER-2013

Mirror Ritual: Human-Machine Co-Construction of Emotion

Mirror Ritual is an interactive installation that challenges the existing paradigms in our understanding of human emotion and machine perception. In contrast to prescriptive interfaces, the work's real-time affective interface engages the audience in the iterative conceptualisation of their emotional state through the use of affectively-charged machine generated poetry. The audience are encouraged to make sense of the mirror's poetry by framing it with respect to their recent life experiences, effectively `putting into words' their felt emotion. This process of affect labelling and contextualisation works to not only regulate emotion, but helps to construct the rich personal narratives that constitute human identity.Comment: Paper presented at ACM TEI Conference 2020 Arts Track, Sydney Australi

Facial Expression Recognition from World Wild Web

Recognizing facial expression in a wild setting has remained a challenging task in computer vision. The World Wide Web is a good source of facial images which most of them are captured in uncontrolled conditions. In fact, the Internet is a Word Wild Web of facial images with expressions. This paper presents the results of a new study on collecting, annotating, and analyzing wild facial expressions from the web. Three search engines were queried using 1250 emotion related keywords in six different languages and the retrieved images were mapped by two annotators to six basic expressions and neutral. Deep neural networks and noise modeling were used in three different training scenarios to find how accurately facial expressions can be recognized when trained on noisy images collected from the web using query terms (e.g. happy face, laughing man, etc)? The results of our experiments show that deep neural networks can recognize wild facial expressions with an accuracy of 82.12%