3 research outputs found
Spontaneous Facial Micro-Expression Recognition using 3D Spatiotemporal Convolutional Neural Networks
Facial expression recognition in videos is an active area of research in
computer vision. However, fake facial expressions are difficult to be
recognized even by humans. On the other hand, facial micro-expressions
generally represent the actual emotion of a person, as it is a spontaneous
reaction expressed through human face. Despite of a few attempts made for
recognizing micro-expressions, still the problem is far from being a solved
problem, which is depicted by the poor rate of accuracy shown by the
state-of-the-art methods. A few CNN based approaches are found in the
literature to recognize micro-facial expressions from still images. Whereas, a
spontaneous micro-expression video contains multiple frames that have to be
processed together to encode both spatial and temporal information. This paper
proposes two 3D-CNN methods: MicroExpSTCNN and MicroExpFuseNet, for spontaneous
facial micro-expression recognition by exploiting the spatiotemporal
information in CNN framework. The MicroExpSTCNN considers the full spatial
information, whereas the MicroExpFuseNet is based on the 3D-CNN feature fusion
of the eyes and mouth regions. The experiments are performed over CAS(ME)^2 and
SMIC micro-expression databases. The proposed MicroExpSTCNN model outperforms
the state-of-the-art methods.Comment: Accepted in 2019 International Joint Conference on Neural Networks
(IJCNN
Spatiotemporal Recurrent Convolutional Networks for Recognizing Spontaneous Micro-expressions
Recently, the recognition task of spontaneous facial micro-expressions has
attracted much attention with its various real-world applications. Plenty of
handcrafted or learned features have been employed for a variety of classifiers
and achieved promising performances for recognizing micro-expressions. However,
the micro-expression recognition is still challenging due to the subtle
spatiotemporal changes of micro-expressions. To exploit the merits of deep
learning, we propose a novel deep recurrent convolutional networks based
micro-expression recognition approach, capturing the spatial-temporal
deformations of micro-expression sequence. Specifically, the proposed deep
model is constituted of several recurrent convolutional layers for extracting
visual features and a classificatory layer for recognition. It is optimized by
an end-to-end manner and obviates manual feature design. To handle sequential
data, we exploit two types of extending the connectivity of convolutional
networks across temporal domain, in which the spatiotemporal deformations are
modeled in views of facial appearance and geometry separately. Besides, to
overcome the shortcomings of limited and imbalanced training samples, temporal
data augmentation strategies as well as a balanced loss are jointly used for
our deep network. By performing the experiments on three spontaneous
micro-expression datasets, we verify the effectiveness of our proposed
micro-expression recognition approach compared to the state-of-the-art methods.Comment: Submitted to IEEE TM
A Review on Facial Micro-Expressions Analysis: Datasets, Features and Metrics
Facial micro-expressions are very brief, spontaneous facial expressions that
appear on the face of humans when they either deliberately or unconsciously
conceal an emotion. Micro-expression has shorter duration than
macro-expression, which makes it more challenging for human and machine. Over
the past ten years, automatic micro-expressions recognition has attracted
increasing attention from researchers in psychology, computer science,
security, neuroscience and other related disciplines. The aim of this paper is
to provide the insights of automatic micro-expressions and recommendations for
future research. There has been a lot of datasets released over the last decade
that facilitated the rapid growth in this field. However, comparison across
different datasets is difficult due to the inconsistency in experiment
protocol, features used and evaluation methods. To address these issues, we
review the datasets, features and the performance metrics deployed in the
literature. Relevant challenges such as the spatial temporal settings during
data collection, emotional classes versus objective classes in data labelling,
face regions in data analysis, standardisation of metrics and the requirements
for real-world implementation are discussed. We conclude by proposing some
promising future directions to advancing micro-expressions research.Comment: Preprint submitted to IEEE Transaction