5,419 research outputs found
Shallow Triple Stream Three-dimensional CNN (STSTNet) for Micro-expression Recognition
In the recent year, state-of-the-art for facial micro-expression recognition
have been significantly advanced by deep neural networks. The robustness of
deep learning has yielded promising performance beyond that of traditional
handcrafted approaches. Most works in literature emphasized on increasing the
depth of networks and employing highly complex objective functions to learn
more features. In this paper, we design a Shallow Triple Stream
Three-dimensional CNN (STSTNet) that is computationally light whilst capable of
extracting discriminative high level features and details of micro-expressions.
The network learns from three optical flow features (i.e., optical strain,
horizontal and vertical optical flow fields) computed based on the onset and
apex frames of each video. Our experimental results demonstrate the
effectiveness of the proposed STSTNet, which obtained an unweighted average
recall rate of 0.7605 and unweighted F1-score of 0.7353 on the composite
database consisting of 442 samples from the SMIC, CASME II and SAMM databases.Comment: 5 pages, 1 figure, Accepted and published in IEEE FG 201
A Novel Apex-Time Network for Cross-Dataset Micro-Expression Recognition
The automatic recognition of micro-expression has been boosted ever since the
successful introduction of deep learning approaches. As researchers working on
such topics are moving to learn from the nature of micro-expression, the
practice of using deep learning techniques has evolved from processing the
entire video clip of micro-expression to the recognition on apex frame. Using
the apex frame is able to get rid of redundant video frames, but the relevant
temporal evidence of micro-expression would be thereby left out. This paper
proposes a novel Apex-Time Network (ATNet) to recognize micro-expression based
on spatial information from the apex frame as well as on temporal information
from the respective-adjacent frames. Through extensive experiments on three
benchmarks, we demonstrate the improvement achieved by learning such temporal
information. Specially, the model with such temporal information is more robust
in cross-dataset validations.Comment: 6 pages, 3 figures, 3 tables, code available, accepted in ACII 201
Data Leakage and Evaluation Issues in Micro-Expression Analysis
Micro-expressions have drawn increasing interest lately due to various
potential applications. The task is, however, difficult as it incorporates many
challenges from the fields of computer vision, machine learning and emotional
sciences. Due to the spontaneous and subtle characteristics of
micro-expressions, the available training and testing data are limited, which
make evaluation complex. We show that data leakage and fragmented evaluation
protocols are issues among the micro-expression literature. We find that fixing
data leaks can drastically reduce model performance, in some cases even making
the models perform similarly to a random classifier. To this end, we go through
common pitfalls, propose a new standardized evaluation protocol using facial
action units with over 2000 micro-expression samples, and provide an open
source library that implements the evaluation protocols in a standardized
manner. Code will be available in \url{https://github.com/tvaranka/meb}
Micro-attention for micro-expression recognition
Micro-expression, for its high objectivity in emotion detection, has emerged to be a promising modality in affective computing. Recently, deep learning methods have been successfully introduced into the micro-expression recognition area. Whilst the higher recognition accuracy achieved, substantial challenges in micro-expression recognition remain. The existence of micro expression in small-local areas on face and limited size of available databases still constrain the recognition accuracy on such emotional facial behavior. In this work, to tackle such challenges, we propose a novel attention mechanism called micro-attention cooperating with residual network. Micro-attention enables the network to learn to focus on facial areas of interest covering different action units. Moreover, coping with small datasets, the micro-attention is designed without adding noticeable parameters while a simple yet efficient transfer learning approach is together utilized to alleviate the overfitting risk. With extensive experimental evaluations on three benchmarks (CASMEII, SAMM and SMIC) and post-hoc feature visualizations, we demonstrate the effectiveness of the proposed micro-attention and push the boundary of automatic recognition of micro-expression
How to Synthesize a Large-Scale and Trainable Micro-Expression Dataset?
This paper does not contain technical novelty but introduces our key
discoveries in a data generation protocol, a database and insights. We aim to
address the lack of large-scale datasets in micro-expression (MiE) recognition
due to the prohibitive cost of data collection, which renders large-scale
training less feasible. To this end, we develop a protocol to automatically
synthesize large scale MiE training data that allow us to train improved
recognition models for real-world test data. Specifically, we discover three
types of Action Units (AUs) that can constitute trainable MiEs. These AUs come
from real-world MiEs, early frames of macro-expression videos, and the
relationship between AUs and expression categories defined by human expert
knowledge. With these AUs, our protocol then employs large numbers of face
images of various identities and an off-the-shelf face generator for MiE
synthesis, yielding the MiE-X dataset. MiE recognition models are trained or
pre-trained on MiE-X and evaluated on real-world test sets, where very
competitive accuracy is obtained. Experimental results not only validate the
effectiveness of the discovered AUs and MiE-X dataset but also reveal some
interesting properties of MiEs: they generalize across faces, are close to
early-stage macro-expressions, and can be manually defined.Comment: European Conference on Computer Vision 202
Time Efficient Micro-Expression Recognition using Weighted Spatio-Temporal Landmark Graphs
Micro-expressions have been shown to be effective in understanding the genuine emotions of a person. While many advances have been made in detecting micro-expressions using deep learning, previous studies in recognizing micro-expressions require pre-processing steps and the use of large feature sets resulting in large runtimes and thus have limited applicability in real-world scenarios. In this paper, we propose time-efficient end-to-end framework which uses landmark-based positional features to generate spatio-temporal graphs that can be applied to micro-expression recognition using Graph Convolutional NeuralNetworks (GCNs). We explore the importance of landmark features and propose a selective feature reduction approach to further improve efficiency. We perform experiments using the SMIC, CASMEII and SAMM datasets and demonstrate that our approach significantly speeds up predictions and delivers resultscomparable to the state-of-the-art
- …