125,114 research outputs found
3d Face Reconstruction And Emotion Analytics With Part-Based Morphable Models
3D face reconstruction and facial expression analytics using 3D facial data are new
and hot research topics in computer graphics and computer vision. In this proposal, we first
review the background knowledge for emotion analytics using 3D morphable face model, including
geometry feature-based methods, statistic model-based methods and more advanced
deep learning-bade methods. Then, we introduce a novel 3D face modeling and reconstruction
solution that robustly and accurately acquires 3D face models from a couple of images
captured by a single smartphone camera. Two selfie photos of a subject taken from the
front and side are used to guide our Non-Negative Matrix Factorization (NMF) induced
part-based face model to iteratively reconstruct an initial 3D face of the subject. Then, an
iterative detail updating method is applied to the initial generated 3D face to reconstruct
facial details through optimizing lighting parameters and local depths. Our iterative 3D
face reconstruction method permits fully automatic registration of a part-based face representation
to the acquired face data and the detailed 2D/3D features to build a high-quality
3D face model. The NMF part-based face representation learned from a 3D face database
facilitates effective global and adaptive local detail data fitting alternatively. Our system
is flexible and it allows users to conduct the capture in any uncontrolled environment. We
demonstrate the capability of our method by allowing users to capture and reconstruct their
3D faces by themselves.
Based on the 3D face model reconstruction, we can analyze the facial expression and
the related emotion in 3D space. We present a novel approach to analyze the facial expressions
from images and a quantitative information visualization scheme for exploring this
type of visual data. From the reconstructed result using NMF part-based morphable 3D face
model, basis parameters and a displacement map are extracted as features for facial emotion
analysis and visualization. Based upon the features, two Support Vector Regressions (SVRs)
are trained to determine the fuzzy Valence-Arousal (VA) values to quantify the emotions.
The continuously changing emotion status can be intuitively analyzed by visualizing the
VA values in VA-space. Our emotion analysis and visualization system, based on 3D NMF
morphable face model, detects expressions robustly from various head poses, face sizes and
lighting conditions, and is fully automatic to compute the VA values from images or a sequence
of video with various facial expressions. To evaluate our novel method, we test our
system on publicly available databases and evaluate the emotion analysis and visualization
results. We also apply our method to quantifying emotion changes during motivational interviews.
These experiments and applications demonstrate effectiveness and accuracy of
our method.
In order to improve the expression recognition accuracy, we present a facial expression
recognition approach with 3D Mesh Convolutional Neural Network (3DMCNN) and a visual
analytics guided 3DMCNN design and optimization scheme. The geometric properties of the
surface is computed using the 3D face model of a subject with facial expressions. Instead of
using regular Convolutional Neural Network (CNN) to learn intensities of the facial images,
we convolve the geometric properties on the surface of the 3D model using 3DMCNN. We
design a geodesic distance-based convolution method to overcome the difficulties raised from
the irregular sampling of the face surface mesh. We further present an interactive visual
analytics for the purpose of designing and modifying the networks to analyze the learned
features and cluster similar nodes in 3DMCNN. By removing low activity nodes in the network,
the performance of the network is greatly improved. We compare our method with the regular CNN-based method by interactively visualizing each layer of the networks and
analyze the effectiveness of our method by studying representative cases. Testing on public
datasets, our method achieves a higher recognition accuracy than traditional image-based
CNN and other 3D CNNs. The presented framework, including 3DMCNN and interactive
visual analytics of the CNN, can be extended to other applications
Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment
Facial action unit (AU) detection and face alignment are two highly
correlated tasks since facial landmarks can provide precise AU locations to
facilitate the extraction of meaningful local features for AU detection. Most
existing AU detection works often treat face alignment as a preprocessing and
handle the two tasks independently. In this paper, we propose a novel
end-to-end deep learning framework for joint AU detection and face alignment,
which has not been explored before. In particular, multi-scale shared features
are learned firstly, and high-level features of face alignment are fed into AU
detection. Moreover, to extract precise local features, we propose an adaptive
attention learning module to refine the attention map of each AU adaptively.
Finally, the assembled local features are integrated with face alignment
features and global features for AU detection. Experiments on BP4D and DISFA
benchmarks demonstrate that our framework significantly outperforms the
state-of-the-art methods for AU detection.Comment: This paper has been accepted by ECCV 201
Automatic Analysis of Facial Expressions Based on Deep Covariance Trajectories
In this paper, we propose a new approach for facial expression recognition
using deep covariance descriptors. The solution is based on the idea of
encoding local and global Deep Convolutional Neural Network (DCNN) features
extracted from still images, in compact local and global covariance
descriptors. The space geometry of the covariance matrices is that of Symmetric
Positive Definite (SPD) matrices. By conducting the classification of static
facial expressions using Support Vector Machine (SVM) with a valid Gaussian
kernel on the SPD manifold, we show that deep covariance descriptors are more
effective than the standard classification with fully connected layers and
softmax. Besides, we propose a completely new and original solution to model
the temporal dynamic of facial expressions as deep trajectories on the SPD
manifold. As an extension of the classification pipeline of covariance
descriptors, we apply SVM with valid positive definite kernels derived from
global alignment for deep covariance trajectories classification. By performing
extensive experiments on the Oulu-CASIA, CK+, and SFEW datasets, we show that
both the proposed static and dynamic approaches achieve state-of-the-art
performance for facial expression recognition outperforming many recent
approaches.Comment: A preliminary version of this work appeared in "Otberdout N, Kacem A,
Daoudi M, Ballihi L, Berretti S. Deep Covariance Descriptors for Facial
Expression Recognition, in British Machine Vision Conference 2018, BMVC 2018,
Northumbria University, Newcastle, UK, September 3-6, 2018. ; 2018 :159."
arXiv admin note: substantial text overlap with arXiv:1805.0386
Facial Point Detection using Boosted Regression and Graph Models
Finding fiducial facial points in any frame of a video showing rich naturalistic facial behaviour is an unsolved problem. Yet this is a crucial step for geometric-featurebased facial expression analysis, and methods that use appearance-based features extracted at fiducial facial point locations. In this paper we present a method based on a combination of Support Vector Regression and Markov Random Fields to drastically reduce the time needed to search for a point’s location and increase the accuracy and robustness of the algorithm. Using Markov Random Fields allows us to constrain the search space by exploiting the constellations that facial points can form. The regressors on the other hand learn a mapping between the appearance of the area surrounding a point and the positions of these points, which makes detection of the points very fast and can make the algorithm robust to variations of appearance due to facial expression and moderate changes in head pose. The proposed point detection algorithm was tested on 1855 images, the results of which showed we outperform current state of the art point detectors
Recommended from our members
Automatic affective dimension recognition from naturalistic facial expressions based on wavelet filtering and PLS regression
Automatic affective dimension recognition from facial expression continuously in naturalistic contexts is a very challenging research topic but very important in human-computer interaction. In this paper, an automatic recognition system was proposed to predict the affective dimensions such as Arousal, Valence and Dominance continuously in naturalistic facial expression videos. Firstly, visual and vocal features are extracted from image frames and audio segments in facial expression videos. Secondly, a wavelet transform based digital filtering method is applied to remove the irrelevant noise information in the feature space. Thirdly, Partial Least Squares regression is used to predict the affective dimensions from both video and audio modalities. Finally, two modalities are combined to boost overall performance in the decision fusion process. The proposed method is tested in the fourth international Audio/Visual Emotion Recognition Challenge (AVEC2014) dataset and compared to other state-of-the-art methods in the affect recognition sub-challenge with a good performance
- …