2,399 research outputs found
Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment
Facial action unit (AU) detection and face alignment are two highly
correlated tasks since facial landmarks can provide precise AU locations to
facilitate the extraction of meaningful local features for AU detection. Most
existing AU detection works often treat face alignment as a preprocessing and
handle the two tasks independently. In this paper, we propose a novel
end-to-end deep learning framework for joint AU detection and face alignment,
which has not been explored before. In particular, multi-scale shared features
are learned firstly, and high-level features of face alignment are fed into AU
detection. Moreover, to extract precise local features, we propose an adaptive
attention learning module to refine the attention map of each AU adaptively.
Finally, the assembled local features are integrated with face alignment
features and global features for AU detection. Experiments on BP4D and DISFA
benchmarks demonstrate that our framework significantly outperforms the
state-of-the-art methods for AU detection.Comment: This paper has been accepted by ECCV 201
Facial Action Unit Detection Using Attention and Relation Learning
Attention mechanism has recently attracted increasing attentions in the field
of facial action unit (AU) detection. By finding the region of interest of each
AU with the attention mechanism, AU-related local features can be captured.
Most of the existing attention based AU detection works use prior knowledge to
predefine fixed attentions or refine the predefined attentions within a small
range, which limits their capacity to model various AUs. In this paper, we
propose an end-to-end deep learning based attention and relation learning
framework for AU detection with only AU labels, which has not been explored
before. In particular, multi-scale features shared by each AU are learned
firstly, and then both channel-wise and spatial attentions are adaptively
learned to select and extract AU-related local features. Moreover, pixel-level
relations for AUs are further captured to refine spatial attentions so as to
extract more relevant local features. Without changing the network
architecture, our framework can be easily extended for AU intensity estimation.
Extensive experiments show that our framework (i) soundly outperforms the
state-of-the-art methods for both AU detection and AU intensity estimation on
the challenging BP4D, DISFA, FERA 2015 and BP4D+ benchmarks, (ii) can
adaptively capture the correlated regions of each AU, and (iii) also works well
under severe occlusions and large poses.Comment: This paper is accepted by IEEE Transactions on Affective Computin
Constrained Joint Cascade Regression Framework for Simultaneous Facial Action Unit Recognition and Facial Landmark Detection
Cascade regression framework has been shown to be effective for facial
landmark detection. It starts from an initial face shape and gradually predicts
the face shape update from the local appearance features to generate the facial
landmark locations in the next iteration until convergence. In this paper, we
improve upon the cascade regression framework and propose the Constrained Joint
Cascade Regression Framework (CJCRF) for simultaneous facial action unit
recognition and facial landmark detection, which are two related face analysis
tasks, but are seldomly exploited together. In particular, we first learn the
relationships among facial action units and face shapes as a constraint. Then,
in the proposed constrained joint cascade regression framework, with the help
from the constraint, we iteratively update the facial landmark locations and
the action unit activation probabilities until convergence. Experimental
results demonstrate that the intertwined relationships of facial action units
and face shapes boost the performances of both facial action unit recognition
and facial landmark detection. The experimental results also demonstrate the
effectiveness of the proposed method comparing to the state-of-the-art works.Comment: International Conference on Computer Vision and Pattern Recognition,
201
Automatic analysis of facial actions: a survey
As one of the most comprehensive and objective ways to describe facial expressions, the Facial Action Coding System (FACS) has recently received significant attention. Over the past 30 years, extensive research has been conducted by psychologists and neuroscientists on various aspects of facial expression analysis using FACS. Automating FACS coding would make this research faster and more widely applicable, opening up new avenues to understanding how we communicate through facial expressions. Such an automated process can also potentially increase the reliability, precision and temporal resolution of coding. This paper provides a comprehensive survey of research into machine analysis of facial actions. We systematically review all components of such systems: pre-processing, feature extraction and machine coding of facial actions. In addition, the existing FACS-coded facial expression databases are summarised. Finally, challenges that have to be addressed to make automatic facial action analysis applicable in real-life situations are extensively discussed. There are two underlying motivations for us to write this survey paper: the first is to provide an up-to-date review of the existing literature, and the second is to offer some insights into the future of machine recognition of facial actions: what are the challenges and opportunities that researchers in the field face
Deep Structure Inference Network for Facial Action Unit Recognition
Facial expressions are combinations of basic components called Action Units
(AU). Recognizing AUs is key for developing general facial expression analysis.
In recent years, most efforts in automatic AU recognition have been dedicated
to learning combinations of local features and to exploiting correlations
between Action Units. In this paper, we propose a deep neural architecture that
tackles both problems by combining learned local and global features in its
initial stages and replicating a message passing algorithm between classes
similar to a graphical model inference approach in later stages. We show that
by training the model end-to-end with increased supervision we improve
state-of-the-art by 5.3% and 8.2% performance on BP4D and DISFA datasets,
respectively
MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Units Detection
The Facial Action Coding System (FACS) encodes the action units (AUs) in
facial images, which has attracted extensive research attention due to its wide
use in facial expression analysis. Many methods that perform well on automatic
facial action unit (AU) detection primarily focus on modeling various types of
AU relations between corresponding local muscle areas, or simply mining global
attention-aware facial features, however, neglect the dynamic interactions
among local-global features. We argue that encoding AU features just from one
perspective may not capture the rich contextual information between regional
and global face features, as well as the detailed variability across AUs,
because of the diversity in expression and individual characteristics. In this
paper, we propose a novel Multi-level Graph Relational Reasoning Network
(termed MGRR-Net) for facial AU detection. Each layer of MGRR-Net performs a
multi-level (i.e., region-level, pixel-wise and channel-wise level) feature
learning. While the region-level feature learning from local face patches
features via graph neural network can encode the correlation across different
AUs, the pixel-wise and channel-wise feature learning via graph attention
network can enhance the discrimination ability of AU features from global face
features. The fused features from the three levels lead to improved AU
discriminative ability. Extensive experiments on DISFA and BP4D AU datasets
show that the proposed approach achieves superior performance than the
state-of-the-art methods.Comment: 10 pages, 4 figures, 8 tables; submitted to IEEE TMM for possible
publication. Copyright may be transferred without notice, after which this
version may no longer be accessibl
- …