5,974 research outputs found
Feature fusion for facial landmark detection: A feature descriptors combination approach
Facial landmark detection is a crucial first step in facial analysis for biometrics and numerous other applications. However, it has proved to be a very challenging task due to the numerous sources of variation in 2D and 3D facial data. Although landmark detection based on descriptors of the 2D and 3D appearance of the face has been extensively studied, the fusion of such feature descriptors is a relatively under-studied issue. In this report, a novel generalized framework for combining facial feature descriptors is presented, and several feature fusion schemes are proposed and evaluated. The proposed framework maps each feature into a similarity score, combines the individual similarity scores into a resultant score, used to select the optimal solution for a queried landmark. The evaluation of the proposed fusion schemes for facial landmark detection clearly indicates that a quadratic distance to similarity mapping in conjunction with a root mean square rule for similarity fusion achieves the best performance in accuracy, efficiency, robustness and monotonicity
Precise Facial Landmark Detection by Reference Heatmap Transformer
Most facial landmark detection methods predict landmarks by mapping the input
facial appearance features to landmark heatmaps and have achieved promising
results. However, when the face image is suffering from large poses, heavy
occlusions and complicated illuminations, they cannot learn discriminative
feature representations and effective facial shape constraints, nor can they
accurately predict the value of each element in the landmark heatmap, limiting
their detection accuracy. To address this problem, we propose a novel Reference
Heatmap Transformer (RHT) by introducing reference heatmap information for more
precise facial landmark detection. The proposed RHT consists of a Soft
Transformation Module (STM) and a Hard Transformation Module (HTM), which can
cooperate with each other to encourage the accurate transformation of the
reference heatmap information and facial shape constraints. Then, a Multi-Scale
Feature Fusion Module (MSFFM) is proposed to fuse the transformed heatmap
features and the semantic features learned from the original face images to
enhance feature representations for producing more accurate target heatmaps. To
the best of our knowledge, this is the first study to explore how to enhance
facial landmark detection by transforming the reference heatmap information.
The experimental results from challenging benchmark datasets demonstrate that
our proposed method outperforms the state-of-the-art methods in the literature.Comment: Accepted by IEEE Transactions on Image Processing, March 202
- …